[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:18:05 dist-test-slave-jq4t krb5kdc[30624](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:18:05 dist-test-slave-jq4t krb5kdc[30624](info): set up 2 sockets
Aug 11 22:18:05 dist-test-slave-jq4t krb5kdc[30624](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 22:18:06.606046 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.388s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:18:06 dist-test-slave-jq4t krb5kdc[30624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:18:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:18:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:18:06.843168 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:41991
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:41991
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:07.204826 30647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:07.205173 30647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:07.205339 30647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:07.214169 30647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:07.214318 30647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:07.214411 30647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:07.214527 30647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:07.214597 30647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:18:07.225409 30647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:41991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:41991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:07.226845 30647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:07.228241 30647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:07.241326 30663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:07.241346 30658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:07.241799 30647 server_base.cc:1034] running on GCE node
W20260811 22:18:07.241603 30657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:07.243078 30647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:07.244716 30647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:07.246066 30647 hybrid_clock.cc:648] HybridClock initialized: now 1786486687246007 us; error 118 us; skew 500 ppm
I20260811 22:18:07.246683 30647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:07 dist-test-slave-jq4t krb5kdc[30624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486687, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:07.250815 30647 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:18:07.253170 30647 webserver.cc:460] Webserver started at http://127.29.229.62:36539/ using document root <none> and password file <none>
I20260811 22:18:07.253983 30647 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:07.254158 30647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:07.254501 30647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:07.258360 30647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "3d7757c2e5d24d9e98be5799ea01b7c5"
format_stamp: "Formatted at 2026-08-11 22:18:07 on dist-test-slave-jq4t"
server_key: "691c2bf9861d66a2dbad3d26c316cd3f"
server_key_iv: "fa61b94e895cbb4eadb5a687db6518dc"
server_key_version: "encryptionkey@0"
I20260811 22:18:07.259703 30647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "3d7757c2e5d24d9e98be5799ea01b7c5"
format_stamp: "Formatted at 2026-08-11 22:18:07 on dist-test-slave-jq4t"
server_key: "691c2bf9861d66a2dbad3d26c316cd3f"
server_key_iv: "fa61b94e895cbb4eadb5a687db6518dc"
server_key_version: "encryptionkey@0"
I20260811 22:18:07.268085 30647 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.001s
I20260811 22:18:07.272857 30672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:07.274847 30647 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:18:07.275275 30647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "3d7757c2e5d24d9e98be5799ea01b7c5"
format_stamp: "Formatted at 2026-08-11 22:18:07 on dist-test-slave-jq4t"
server_key: "691c2bf9861d66a2dbad3d26c316cd3f"
server_key_iv: "fa61b94e895cbb4eadb5a687db6518dc"
server_key_version: "encryptionkey@0"
I20260811 22:18:07.275579 30647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:07.307627 30647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:07.312915 30647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:07.313256 30647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:07.334153 30647 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:41991
I20260811 22:18:07.334199 30747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:41991 every 8 connection(s)
I20260811 22:18:07.335630 30647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:07.337622 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 30647
I20260811 22:18:07.338097 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:18:07.340993 30612 external_mini_cluster.cc:1428] Setting key 433601d3ac374c88f187170ce93ce715
I20260811 22:18:07.341203 30749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:07.354185 30749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5: Bootstrap starting.
I20260811 22:18:07.358417 30749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:07.360255 30749 log.cc:824] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:07.364070 30749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5: No bootstrap required, opened a new log
I20260811 22:18:07.372874 30749 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7757c2e5d24d9e98be5799ea01b7c5" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41991 } }
Aug 11 22:18:07 dist-test-slave-jq4t krb5kdc[30624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:07.373409 30749 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:07.373524 30749 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d7757c2e5d24d9e98be5799ea01b7c5, State: Initialized, Role: FOLLOWER
I20260811 22:18:07.374385 30749 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7757c2e5d24d9e98be5799ea01b7c5" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41991 } }
I20260811 22:18:07.374714 30749 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:18:07.374899 30749 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:18:07.375110 30749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:07.378165 30749 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7757c2e5d24d9e98be5799ea01b7c5" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41991 } }
I20260811 22:18:07.378808 30749 leader_election.cc:302] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d7757c2e5d24d9e98be5799ea01b7c5; no voters:
I20260811 22:18:07.379602 30749 leader_election.cc:288] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:18:07.379731 30759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:07.388631 30759 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5 [term 1 LEADER]: Becoming Leader. State: Replica: 3d7757c2e5d24d9e98be5799ea01b7c5, State: Running, Role: LEADER
I20260811 22:18:07.389294 30759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7757c2e5d24d9e98be5799ea01b7c5" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41991 } }
I20260811 22:18:07.389942 30749 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:07.391247 30752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:07.346198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38728 (local address 127.29.229.62:41991)
0811 22:18:07.347229 (+ 1031us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:07.347247 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:07.348628 (+ 1381us) server_negotiation.cc:407] Connection header received
0811 22:18:07.358618 (+ 9990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:07.358649 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:07.359244 (+ 595us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:07.359740 (+ 496us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:07.363259 (+ 3519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:07.365401 (+ 2142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:07.367403 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:07.368511 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:07.368721 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:07.375335 (+ 6614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:07.375346 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:07.375362 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:07.375411 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:07.379022 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:07.381471 (+ 2449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:07.381482 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:07.381493 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:07.381904 (+ 411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:07.382920 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:07.383045 (+ 125us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:07.383054 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:07.384398 (+ 1344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:07.385386 (+ 988us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:07.389158 (+ 3772us) server_negotiation.cc:299] Negotiation successful
0811 22:18:07.389613 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":453,"threads_started":1}
I20260811 22:18:07.392854 30762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d7757c2e5d24d9e98be5799ea01b7c5. Latest consensus state: current_term: 1 leader_uuid: "3d7757c2e5d24d9e98be5799ea01b7c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7757c2e5d24d9e98be5799ea01b7c5" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41991 } } }
I20260811 22:18:07.393318 30762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:07.393883 30761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d7757c2e5d24d9e98be5799ea01b7c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7757c2e5d24d9e98be5799ea01b7c5" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41991 } } }
I20260811 22:18:07.394237 30761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:07.396504 30775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:07.402812 30775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:07.412117 30775 catalog_manager.cc:1347] Generated new cluster ID: 6467b923d530499e986c8887d1ff5ab7
I20260811 22:18:07.412292 30775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:07.428398 30775 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:18:07.431316 30775 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:07.442190 30775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d7757c2e5d24d9e98be5799ea01b7c5: Generated new TSK 0
I20260811 22:18:07.443181 30775 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
I20260811 22:18:07.501135 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41991
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:07.871390 30794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:07.871673 30794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:07.871857 30794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:07.879729 30794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:07.880065 30794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:07.880288 30794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:18:07.891098 30794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41991
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:07.892680 30794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:07.894053 30794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:07.907902 30805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:07.907925 30807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:07.907954 30804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:07.909073 30794 server_base.cc:1034] running on GCE node
I20260811 22:18:07.909690 30794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:07.910574 30794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:07.911791 30794 hybrid_clock.cc:648] HybridClock initialized: now 1786486687911737 us; error 44 us; skew 500 ppm
I20260811 22:18:07.912089 30794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:07 dist-test-slave-jq4t krb5kdc[30624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486687, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:07.915485 30794 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:18:07.917289 30794 webserver.cc:460] Webserver started at http://127.29.229.1:39337/ using document root <none> and password file <none>
I20260811 22:18:07.917974 30794 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:07.918087 30794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:07.918354 30794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:07.921465 30794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "5d85ad7be5924266a71d9d31910b485e"
format_stamp: "Formatted at 2026-08-11 22:18:07 on dist-test-slave-jq4t"
server_key: "7ab0ecadd7cb378a3cb99ae80139f861"
server_key_iv: "73a5831a0dced3b6850272be10934f38"
server_key_version: "encryptionkey@0"
I20260811 22:18:07.922260 30794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "5d85ad7be5924266a71d9d31910b485e"
format_stamp: "Formatted at 2026-08-11 22:18:07 on dist-test-slave-jq4t"
server_key: "7ab0ecadd7cb378a3cb99ae80139f861"
server_key_iv: "73a5831a0dced3b6850272be10934f38"
server_key_version: "encryptionkey@0"
I20260811 22:18:07.927735 30794 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:18:07.931587 30815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:07.933105 30794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 22:18:07.933252 30794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "5d85ad7be5924266a71d9d31910b485e"
format_stamp: "Formatted at 2026-08-11 22:18:07 on dist-test-slave-jq4t"
server_key: "7ab0ecadd7cb378a3cb99ae80139f861"
server_key_iv: "73a5831a0dced3b6850272be10934f38"
server_key_version: "encryptionkey@0"
I20260811 22:18:07.933429 30794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:07.958011 30794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:07.962710 30794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:07.962980 30794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:07.963974 30794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:07.965471 30794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:07.965559 30794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:07.965646 30794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:07.965693 30794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:08.006004 30794 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:45241
I20260811 22:18:08.006100 31008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:45241 every 8 connection(s)
I20260811 22:18:08.007503 30794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:18:08.017756 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 30794
I20260811 22:18:08.018018 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:18:08.018819 30612 external_mini_cluster.cc:1428] Setting key 509ac687fde11da01693b0c22b13d24b
Aug 11 22:18:08 dist-test-slave-jq4t krb5kdc[30624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486687, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:08.031009 31013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:08.010297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:46531 (local address 127.29.229.62:41991)
0811 22:18:08.010815 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:08.010820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:08.011682 (+ 862us) server_negotiation.cc:407] Connection header received
0811 22:18:08.013765 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:08.013773 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:08.013878 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:08.014001 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:08.015629 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:08.016375 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:08.018580 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:08.019132 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:08.019299 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:08.024262 (+ 4963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:08.024268 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:08.024271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:08.024320 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:08.027261 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:08.028114 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:08.028120 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:08.028125 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:08.028255 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:08.028804 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:08.028809 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:08.028811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:08.029441 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:08.029665 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:08.030473 (+ 808us) server_negotiation.cc:299] Negotiation successful
0811 22:18:08.030694 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":292,"threads_started":1}
I20260811 22:18:08.032207 31016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:08.010719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41991 (local address 127.29.229.1:46531)
0811 22:18:08.011511 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:08.011558 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:08.013473 (+ 1915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:08.014255 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:08.014263 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:08.014848 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:08.015448 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:08.015466 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:08.016582 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:08.016587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:08.018377 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:08.018394 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:08.019440 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:08.019448 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:08.019711 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:08.020800 (+ 1089us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:08.020837 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:08.023982 (+ 3145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:08.027447 (+ 3465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:08.027457 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:08.027475 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:08.027973 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:08.028373 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:08.028379 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:08.028383 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:08.028671 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:08.029626 (+ 955us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:08.029633 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:08.029997 (+ 364us) client_negotiation.cc:769] Sending connection context
0811 22:18:08.030276 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 22:18:08.030725 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":317,"threads_started":1}
I20260811 22:18:08.033988 31009 heartbeater.cc:347] Connected to a master server at 127.29.229.62:41991
I20260811 22:18:08.034349 31009 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:08.035161 31009 heartbeater.cc:511] Master 127.29.229.62:41991 requested a full tablet report, sending...
I20260811 22:18:08.037851 30695 ts_manager.cc:194] Registered new tserver with Master: 5d85ad7be5924266a71d9d31910b485e (127.29.229.1:45241)
I20260811 22:18:08.039588 30695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:46531
WARNING: no policy specified for kudu/127.29.229.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
I20260811 22:18:08.076887 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41991
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:08.436050 31019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:08.436362 31019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:08.436604 31019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:08.445299 31019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:08.445418 31019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:08.445541 31019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:18:08.456431 31019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41991
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:08.458012 31019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:08.459357 31019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:08.473804 31026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:08.473762 31027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:08.473783 31032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:08.474776 31019 server_base.cc:1034] running on GCE node
I20260811 22:18:08.475368 31019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:08.476387 31019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:08.477599 31019 hybrid_clock.cc:648] HybridClock initialized: now 1786486688477561 us; error 35 us; skew 500 ppm
I20260811 22:18:08.477941 31019 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:08 dist-test-slave-jq4t krb5kdc[30624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486688, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:08.481251 31019 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:18:08.482726 31019 webserver.cc:460] Webserver started at http://127.29.229.2:37849/ using document root <none> and password file <none>
I20260811 22:18:08.483450 31019 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:08.483610 31019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:08.483959 31019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:08.487010 31019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "53d464a5bc7744d3ad510e4f62c421fe"
format_stamp: "Formatted at 2026-08-11 22:18:08 on dist-test-slave-jq4t"
server_key: "38ccf59d29c39772b2115c2dae3749bc"
server_key_iv: "9c4cf84b3afd2917ec69c8106fab2e94"
server_key_version: "encryptionkey@0"
I20260811 22:18:08.487821 31019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "53d464a5bc7744d3ad510e4f62c421fe"
format_stamp: "Formatted at 2026-08-11 22:18:08 on dist-test-slave-jq4t"
server_key: "38ccf59d29c39772b2115c2dae3749bc"
server_key_iv: "9c4cf84b3afd2917ec69c8106fab2e94"
server_key_version: "encryptionkey@0"
I20260811 22:18:08.494350 31019 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.001s
I20260811 22:18:08.498277 31040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:08.499614 31019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:18:08.500006 31019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "53d464a5bc7744d3ad510e4f62c421fe"
format_stamp: "Formatted at 2026-08-11 22:18:08 on dist-test-slave-jq4t"
server_key: "38ccf59d29c39772b2115c2dae3749bc"
server_key_iv: "9c4cf84b3afd2917ec69c8106fab2e94"
server_key_version: "encryptionkey@0"
I20260811 22:18:08.500259 31019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:08.538105 31019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:08.542608 31019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:08.542928 31019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:08.544130 31019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:08.545914 31019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:08.546038 31019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:08.546224 31019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:08.546319 31019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:08.586637 31019 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:38127
I20260811 22:18:08.586702 31218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:38127 every 8 connection(s)
I20260811 22:18:08.588140 31019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:18:08.591766 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 31019
I20260811 22:18:08.592252 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:18:08.593384 30612 external_mini_cluster.cc:1428] Setting key 12e6dfb703e9bd58983b7607841d6396
Aug 11 22:18:08 dist-test-slave-jq4t krb5kdc[30624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486688, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:08.611784 31223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:08.591200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:48151 (local address 127.29.229.62:41991)
0811 22:18:08.591766 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:08.591774 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:08.592977 (+ 1203us) server_negotiation.cc:407] Connection header received
0811 22:18:08.595164 (+ 2187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:08.595171 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:08.595280 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:08.595406 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:08.597298 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:08.598108 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:08.599717 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:08.600162 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:08.600298 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:08.603999 (+ 3701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:08.604012 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:08.604016 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:08.604050 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:08.607917 (+ 3867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:08.608801 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:08.608817 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:08.608824 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:08.608993 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:08.609542 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:08.609550 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:08.609555 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:08.610025 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:08.610199 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:08.610997 (+ 798us) server_negotiation.cc:299] Negotiation successful
0811 22:18:08.611257 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":262,"threads_started":1}
I20260811 22:18:08.612640 31224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:08.591780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41991 (local address 127.29.229.2:48151)
0811 22:18:08.592739 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:08.592788 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:08.594860 (+ 2072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:08.595667 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:08.595676 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:08.596443 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:08.597110 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:08.597128 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:08.598278 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:08.598288 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:08.599565 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:08.599578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:08.600434 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:08.600445 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:08.600717 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:08.601451 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:08.601481 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:08.603745 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:08.608088 (+ 4343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:08.608095 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:08.608108 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:08.608662 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:08.609148 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:08.609153 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:08.609156 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:08.609398 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:08.610182 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:08.610188 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:08.610558 (+ 370us) client_negotiation.cc:769] Sending connection context
0811 22:18:08.610857 (+ 299us) client_negotiation.cc:240] Negotiation successful
0811 22:18:08.611319 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":424,"threads_started":1}
I20260811 22:18:08.614362 31220 heartbeater.cc:347] Connected to a master server at 127.29.229.62:41991
I20260811 22:18:08.614734 31220 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:08.615765 31220 heartbeater.cc:511] Master 127.29.229.62:41991 requested a full tablet report, sending...
I20260811 22:18:08.617940 30695 ts_manager.cc:194] Registered new tserver with Master: 53d464a5bc7744d3ad510e4f62c421fe (127.29.229.2:38127)
WARNING: no policy specified for kudu/127.29.229.3@KRBTEST.COM; defaulting to no policy
I20260811 22:18:08.619096 30695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:48151
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
I20260811 22:18:08.652230 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41991
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:09.018591 31227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:09.018972 31227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:09.019162 31227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:09.026752 31227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:09.026939 31227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:09.027129 31227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:18:09.037956 31227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41991
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:09.039534 31227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:09.041088 31227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:18:09.043870 31009 heartbeater.cc:503] Master 127.29.229.62:41991 was elected leader, sending a full tablet report...
W20260811 22:18:09.055558 31233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:09.055537 31236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:09.056137 31234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:09.058058 31227 server_base.cc:1034] running on GCE node
I20260811 22:18:09.058844 31227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:09.059921 31227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:09.061199 31227 hybrid_clock.cc:648] HybridClock initialized: now 1786486689061131 us; error 61 us; skew 500 ppm
I20260811 22:18:09.061553 31227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:09 dist-test-slave-jq4t krb5kdc[30624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486689, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:09.065302 31227 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:18:09.066978 31227 webserver.cc:460] Webserver started at http://127.29.229.3:36417/ using document root <none> and password file <none>
I20260811 22:18:09.068466 31227 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:09.068749 31227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:09.069388 31227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:09.072772 31227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "30f0423a8a07422185bb51c891369849"
format_stamp: "Formatted at 2026-08-11 22:18:09 on dist-test-slave-jq4t"
server_key: "f5eba73f2ba7f81275c9aa257ae45055"
server_key_iv: "de56dbca1382775a6d2f4b418712a02d"
server_key_version: "encryptionkey@0"
I20260811 22:18:09.074029 31227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "30f0423a8a07422185bb51c891369849"
format_stamp: "Formatted at 2026-08-11 22:18:09 on dist-test-slave-jq4t"
server_key: "f5eba73f2ba7f81275c9aa257ae45055"
server_key_iv: "de56dbca1382775a6d2f4b418712a02d"
server_key_version: "encryptionkey@0"
I20260811 22:18:09.080899 31227 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 22:18:09.084842 31248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:09.086567 31227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:18:09.086805 31227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "30f0423a8a07422185bb51c891369849"
format_stamp: "Formatted at 2026-08-11 22:18:09 on dist-test-slave-jq4t"
server_key: "f5eba73f2ba7f81275c9aa257ae45055"
server_key_iv: "de56dbca1382775a6d2f4b418712a02d"
server_key_version: "encryptionkey@0"
I20260811 22:18:09.087081 31227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:09.108811 31227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:09.114621 31227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:09.114959 31227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:09.116041 31227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:09.117592 31227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:09.117679 31227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:09.117825 31227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:09.117883 31227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:09.157003 31227 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:46045
I20260811 22:18:09.157047 31436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:46045 every 8 connection(s)
I20260811 22:18:09.158336 31227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:18:09.167073 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 31227
I20260811 22:18:09.167284 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:18:09.167759 30612 external_mini_cluster.cc:1428] Setting key dfc18d15018dd2385fe3800f50ce7a7f
Aug 11 22:18:09 dist-test-slave-jq4t krb5kdc[30624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486689, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:09.180672 31445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:09.161584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:48051 (local address 127.29.229.62:41991)
0811 22:18:09.162088 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:09.162095 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:09.163079 (+ 984us) server_negotiation.cc:407] Connection header received
0811 22:18:09.164988 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:09.164992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:09.165085 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:09.165200 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:09.166699 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:09.167649 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:09.169563 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:09.170253 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:09.170405 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:09.174428 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:09.174433 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:09.174437 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:09.174472 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:09.177359 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:09.178149 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:09.178153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:09.178156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:09.178291 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:09.178815 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:09.178818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:09.178820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:09.179121 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:09.179248 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:09.180084 (+ 836us) server_negotiation.cc:299] Negotiation successful
0811 22:18:09.180268 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":227,"threads_started":1}
I20260811 22:18:09.181648 31446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:09.162108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41991 (local address 127.29.229.3:48051)
0811 22:18:09.162905 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:09.162950 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:09.164715 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:09.165426 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:09.165434 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:09.165961 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:09.166523 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:09.166540 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:09.167828 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:09.167833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:09.169391 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:09.169411 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:09.170509 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:09.170515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:09.170888 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:09.171722 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:09.171750 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:09.174207 (+ 2457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:09.177517 (+ 3310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:09.177524 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:09.177537 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:09.177994 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:09.178427 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:09.178431 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:09.178434 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:09.178701 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:09.179278 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:09.179284 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:09.179649 (+ 365us) client_negotiation.cc:769] Sending connection context
0811 22:18:09.179976 (+ 327us) client_negotiation.cc:240] Negotiation successful
0811 22:18:09.180452 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":315,"threads_started":1}
I20260811 22:18:09.183177 31442 heartbeater.cc:347] Connected to a master server at 127.29.229.62:41991
I20260811 22:18:09.183538 31442 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:09.184439 31442 heartbeater.cc:511] Master 127.29.229.62:41991 requested a full tablet report, sending...
I20260811 22:18:09.185976 30695 ts_manager.cc:194] Registered new tserver with Master: 30f0423a8a07422185bb51c891369849 (127.29.229.3:46045)
I20260811 22:18:09.186841 30695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:48051
I20260811 22:18:09.196990 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:18:09 dist-test-slave-jq4t krb5kdc[30624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486689, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:18:09 dist-test-slave-jq4t krb5kdc[30624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486689, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:18:09.254145 31461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:09.237354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46360 (local address 127.29.229.1:45241)
0811 22:18:09.238144 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:09.238152 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:09.238181 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:18:09.238250 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:09.238256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:09.238475 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:09.238626 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:09.240241 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:09.241416 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:09.242751 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:09.243362 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:09.243467 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:09.247766 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:09.247780 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:09.247795 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:09.247832 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:09.250958 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:09.251698 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:09.251706 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:09.251709 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:09.251843 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:09.252292 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:09.252297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:09.252300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:09.252738 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:09.253096 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:09.253439 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 22:18:09.253742 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":477,"threads_started":1}
W20260811 22:18:09.255483 30856 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46360
Aug 11 22:18:09 dist-test-slave-jq4t krb5kdc[30624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:18:09 dist-test-slave-jq4t krb5kdc[30624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:18:09.314256 31461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:09.296951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46368 (local address 127.29.229.1:45241)
0811 22:18:09.297345 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:09.297353 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:09.297597 (+ 244us) server_negotiation.cc:407] Connection header received
0811 22:18:09.297827 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:09.297831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:09.297939 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:09.298059 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:09.300201 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:09.301364 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:09.302957 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:09.303470 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:09.303629 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:09.308451 (+ 4822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:09.308457 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:09.308460 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:09.308556 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:09.311378 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:09.312103 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:09.312109 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:09.312114 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:09.312256 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:09.312712 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:09.312717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:09.312722 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:09.313012 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:09.313269 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:09.313640 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 22:18:09.313886 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:18:09.315936 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 30794
I20260811 22:18:09.329608 31461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:09.316959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46376 (local address 127.29.229.1:45241)
0811 22:18:09.317176 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:09.317184 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:09.317501 (+ 317us) server_negotiation.cc:407] Connection header received
0811 22:18:09.317684 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:09.317696 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:09.317753 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:09.317856 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:09.319043 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:09.319801 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:09.321164 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:09.321599 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:09.321738 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:09.324114 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:09.324119 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:09.324122 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:09.324153 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:09.326814 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:09.327532 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:09.327538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:09.327542 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:09.327697 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:09.328091 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:09.328099 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:09.328103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:09.328387 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:09.328687 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:09.328997 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 22:18:09.329221 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:18:09.330487 31004 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:09.425356 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 30794
I20260811 22:18:09.450659 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 31019
Aug 11 22:18:09 dist-test-slave-jq4t krb5kdc[30624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.2@KRBTEST.COM
I20260811 22:18:09.469099 31474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:09.451960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40638 (local address 127.29.229.2:38127)
0811 22:18:09.452701 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:09.452709 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:09.452734 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:18:09.452804 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:09.452815 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:09.453047 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:09.453189 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:09.455040 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:09.456110 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:09.457643 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:09.458232 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:09.458351 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:09.462614 (+ 4263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:09.462628 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:09.462642 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:09.462683 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:09.465999 (+ 3316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:09.466705 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:09.466712 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:09.466714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:09.466865 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:09.467329 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:09.467334 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:09.467336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:09.467747 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:09.468006 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:09.468452 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 22:18:09.468767 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":349,"threads_started":1}
I20260811 22:18:09.470902 31212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:09.559062 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 31019
I20260811 22:18:09.586573 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 31227
Aug 11 22:18:09 dist-test-slave-jq4t krb5kdc[30624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.3@KRBTEST.COM
I20260811 22:18:09.606577 31476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:09.587781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38456 (local address 127.29.229.3:46045)
0811 22:18:09.588819 (+ 1038us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:09.588828 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:09.588877 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:18:09.588952 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:09.588959 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:09.589184 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:09.589356 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:09.591088 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:09.592339 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:09.593887 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:09.594521 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:09.594638 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:09.599712 (+ 5074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:09.599729 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:09.599746 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:09.599793 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:09.603038 (+ 3245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:09.603823 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:09.603831 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:09.603833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:09.604000 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:09.604418 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:09.604423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:09.604426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:09.605137 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:09.605501 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:09.605867 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 22:18:09.606182 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":410,"threads_started":1}
I20260811 22:18:09.607940 31427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:09.698427 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 31227
I20260811 22:18:09.727479 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 30647
I20260811 22:18:09.728722 30742 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:09.839118 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 30647
2026-08-11T22:18:09Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4690 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:18:09 dist-test-slave-jq4t krb5kdc[31483](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:18:09 dist-test-slave-jq4t krb5kdc[31483](info): set up 2 sockets
Aug 11 22:18:09 dist-test-slave-jq4t krb5kdc[31483](info): commencing operation
krb5kdc: starting...
W20260811 22:18:11.223489 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.327s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:18:11 dist-test-slave-jq4t krb5kdc[31483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:18:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:18:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:18:11.476234 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:42131
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:37139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:42131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:11.836447 31506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:11.836787 31506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:11.836947 31506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:11.844439 31506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:11.844573 31506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:11.844681 31506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:11.844787 31506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:11.844870 31506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:18:11.854996 31506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:37139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:42131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:42131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:11.856469 31506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:11.857919 31506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:11.872069 31515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:11.872107 31514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:11.871994 31523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:11.872942 31506 server_base.cc:1034] running on GCE node
I20260811 22:18:11.873853 31506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:11.876049 31506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:11.877413 31506 hybrid_clock.cc:648] HybridClock initialized: now 1786486691877326 us; error 44 us; skew 500 ppm
I20260811 22:18:11.878118 31506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:11 dist-test-slave-jq4t krb5kdc[31483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486691, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:11.881866 31506 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:18:11.883726 31506 webserver.cc:460] Webserver started at http://127.29.229.62:32789/ using document root <none> and password file <none>
I20260811 22:18:11.884596 31506 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:11.884760 31506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:11.885140 31506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:11.888422 31506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "9240671f319346a1a58aabb61757e76e"
format_stamp: "Formatted at 2026-08-11 22:18:11 on dist-test-slave-jq4t"
server_key: "478f9f9aaac4ae891e42bfc56b7d6967"
server_key_iv: "30bc79eb24c815c390773631153c2b0c"
server_key_version: "encryptionkey@0"
I20260811 22:18:11.889508 31506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "9240671f319346a1a58aabb61757e76e"
format_stamp: "Formatted at 2026-08-11 22:18:11 on dist-test-slave-jq4t"
server_key: "478f9f9aaac4ae891e42bfc56b7d6967"
server_key_iv: "30bc79eb24c815c390773631153c2b0c"
server_key_version: "encryptionkey@0"
I20260811 22:18:11.895303 31506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:18:11.899716 31530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:11.901609 31506 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 22:18:11.901859 31506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "9240671f319346a1a58aabb61757e76e"
format_stamp: "Formatted at 2026-08-11 22:18:11 on dist-test-slave-jq4t"
server_key: "478f9f9aaac4ae891e42bfc56b7d6967"
server_key_iv: "30bc79eb24c815c390773631153c2b0c"
server_key_version: "encryptionkey@0"
I20260811 22:18:11.902129 31506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:11.918926 31506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:11.923936 31506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:11.924269 31506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:11.946453 31506 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:42131
I20260811 22:18:11.946486 31614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:42131 every 8 connection(s)
I20260811 22:18:11.947942 31506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:11.952178 31617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:11.952070 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 31506
I20260811 22:18:11.952597 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:18:11.953640 30612 external_mini_cluster.cc:1428] Setting key 6da5b5b080ee84a3346895ef4157434d
I20260811 22:18:11.967803 31617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e: Bootstrap starting.
Aug 11 22:18:11 dist-test-slave-jq4t krb5kdc[31483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:11.973009 31617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:11.974188 31617 log.cc:824] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:11.977385 31617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e: No bootstrap required, opened a new log
I20260811 22:18:11.980640 31622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:11.956179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50518 (local address 127.29.229.62:42131)
0811 22:18:11.957176 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:11.957189 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:11.957234 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:18:11.958988 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:11.959014 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:11.959433 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:11.959808 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:11.961821 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:11.963135 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:11.965032 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:11.965770 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:11.965918 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:11.971912 (+ 5994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:11.971920 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:11.971936 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:11.971980 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:11.975453 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:11.976848 (+ 1395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:11.976863 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:11.976870 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:11.977087 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:11.977559 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:11.977570 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:11.977572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:11.978142 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:11.978492 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:11.979026 (+ 534us) server_negotiation.cc:299] Negotiation successful
0811 22:18:11.979504 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":760,"thread_start_us":479,"threads_started":1}
I20260811 22:18:11.986199 31617 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9240671f319346a1a58aabb61757e76e" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42131 } }
I20260811 22:18:11.986634 31617 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:11.986752 31617 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9240671f319346a1a58aabb61757e76e, State: Initialized, Role: FOLLOWER
I20260811 22:18:11.987332 31617 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9240671f319346a1a58aabb61757e76e" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42131 } }
I20260811 22:18:11.987609 31617 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:18:11.987766 31617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:18:11.987982 31617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:11.990656 31617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9240671f319346a1a58aabb61757e76e" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42131 } }
I20260811 22:18:11.991226 31617 leader_election.cc:302] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9240671f319346a1a58aabb61757e76e; no voters:
I20260811 22:18:11.991889 31617 leader_election.cc:288] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:18:11.992090 31624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:11.992880 31624 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e [term 1 LEADER]: Becoming Leader. State: Replica: 9240671f319346a1a58aabb61757e76e, State: Running, Role: LEADER
I20260811 22:18:11.993441 31624 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9240671f319346a1a58aabb61757e76e" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42131 } }
I20260811 22:18:11.994153 31617 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:11.996369 31627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9240671f319346a1a58aabb61757e76e. Latest consensus state: current_term: 1 leader_uuid: "9240671f319346a1a58aabb61757e76e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9240671f319346a1a58aabb61757e76e" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42131 } } }
I20260811 22:18:11.996819 31627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:11.997639 31626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9240671f319346a1a58aabb61757e76e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9240671f319346a1a58aabb61757e76e" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42131 } } }
I20260811 22:18:11.997884 31632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:11.997951 31626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:12.005390 31632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:12.015537 31632 catalog_manager.cc:1347] Generated new cluster ID: 09b6f22069594d45adfed2148d23f1e7
I20260811 22:18:12.015688 31632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:12.048506 31632 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:18:12.049945 31632 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:12.058967 31632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9240671f319346a1a58aabb61757e76e: Generated new TSK 0
I20260811 22:18:12.059794 31632 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
I20260811 22:18:12.116765 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:42131
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:37139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:12.478440 31657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:12.478732 31657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:12.478892 31657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:12.486353 31657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:12.486465 31657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:12.486658 31657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:18:12.497014 31657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:37139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:42131
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:12.498368 31657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:12.499753 31657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:12.514142 31664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:12.514081 31668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:12.514166 31666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:12.514990 31657 server_base.cc:1034] running on GCE node
I20260811 22:18:12.515646 31657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:12.516700 31657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:12.517969 31657 hybrid_clock.cc:648] HybridClock initialized: now 1786486692517904 us; error 65 us; skew 500 ppm
I20260811 22:18:12.518327 31657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:12 dist-test-slave-jq4t krb5kdc[31483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486692, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:12.521708 31657 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:18:12.523285 31657 webserver.cc:460] Webserver started at http://127.29.229.1:44387/ using document root <none> and password file <none>
I20260811 22:18:12.524026 31657 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:12.524173 31657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:12.524554 31657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:12.527594 31657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "fb39f83ab39948848ca58f3b63bc27bf"
format_stamp: "Formatted at 2026-08-11 22:18:12 on dist-test-slave-jq4t"
server_key: "a7b093367ced0e6b03c161135f19cdaf"
server_key_iv: "a44a19457d615e21339d21d06230902f"
server_key_version: "encryptionkey@0"
I20260811 22:18:12.528429 31657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "fb39f83ab39948848ca58f3b63bc27bf"
format_stamp: "Formatted at 2026-08-11 22:18:12 on dist-test-slave-jq4t"
server_key: "a7b093367ced0e6b03c161135f19cdaf"
server_key_iv: "a44a19457d615e21339d21d06230902f"
server_key_version: "encryptionkey@0"
I20260811 22:18:12.533823 31657 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 22:18:12.537554 31682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:12.538915 31657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:18:12.539124 31657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "fb39f83ab39948848ca58f3b63bc27bf"
format_stamp: "Formatted at 2026-08-11 22:18:12 on dist-test-slave-jq4t"
server_key: "a7b093367ced0e6b03c161135f19cdaf"
server_key_iv: "a44a19457d615e21339d21d06230902f"
server_key_version: "encryptionkey@0"
I20260811 22:18:12.539341 31657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:12.564018 31657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:12.568666 31657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:12.569002 31657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:12.570071 31657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:12.571728 31657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:12.571836 31657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:12.571998 31657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:12.572073 31657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:12.609310 31657 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:36757
I20260811 22:18:12.609409 31857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:36757 every 8 connection(s)
I20260811 22:18:12.610682 31657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:18:12.612761 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 31657
I20260811 22:18:12.613011 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:18:12.613901 30612 external_mini_cluster.cc:1428] Setting key 8d9ab91c56c7244129eb4b397533e785
Aug 11 22:18:12 dist-test-slave-jq4t krb5kdc[31483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486692, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:12.632434 31860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:12.613806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:56071 (local address 127.29.229.62:42131)
0811 22:18:12.614960 (+ 1154us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:12.614969 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:12.615565 (+ 596us) server_negotiation.cc:407] Connection header received
0811 22:18:12.617630 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:12.617635 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:12.617731 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:12.617865 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:12.619416 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:12.620160 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:12.621859 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:12.622280 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:12.622428 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:12.626074 (+ 3646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:12.626079 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:12.626082 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:12.626116 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:12.628894 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:12.629687 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:12.629692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:12.629694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:12.629827 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:12.630297 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:12.630301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:12.630303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:12.630589 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:12.630767 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:12.631802 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0811 22:18:12.632055 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":973,"thread_start_us":843,"threads_started":1}
I20260811 22:18:12.633549 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:12.614477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:42131 (local address 127.29.229.1:56071)
0811 22:18:12.615410 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:12.615459 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:12.617360 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:12.618106 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:12.618115 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:12.618650 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:12.619235 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:12.619253 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:12.620344 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:12.620355 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:12.621685 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:12.621699 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:12.622546 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:12.622554 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:12.622786 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:12.623510 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:12.623533 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:12.625837 (+ 2304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:12.629046 (+ 3209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:12.629056 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:12.629073 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:12.629541 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:12.629952 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:12.629958 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:12.629961 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:12.630158 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:12.630729 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:12.630738 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:12.631369 (+ 631us) client_negotiation.cc:769] Sending connection context
0811 22:18:12.631665 (+ 296us) client_negotiation.cc:240] Negotiation successful
0811 22:18:12.632258 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":387,"threads_started":1}
WARNING: no policy specified for kudu/127.29.229.2@KRBTEST.COM; defaulting to no policy
I20260811 22:18:12.635502 31858 heartbeater.cc:347] Connected to a master server at 127.29.229.62:42131
I20260811 22:18:12.635797 31858 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:12.636731 31858 heartbeater.cc:511] Master 127.29.229.62:42131 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.2@KRBTEST.COM" created.
I20260811 22:18:12.639307 31560 ts_manager.cc:194] Registered new tserver with Master: fb39f83ab39948848ca58f3b63bc27bf (127.29.229.1:36757)
I20260811 22:18:12.641580 31560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:56071
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
I20260811 22:18:12.666256 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:42131
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:37139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:13.027212 31868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:13.027609 31868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:13.027799 31868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:13.035324 31868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:13.035507 31868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:13.035697 31868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:18:13.046437 31868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:37139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:42131
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:13.047943 31868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:13.049547 31868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:13.063565 31874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:13.063587 31875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:13.063558 31878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:13.069974 31868 server_base.cc:1034] running on GCE node
I20260811 22:18:13.070734 31868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:13.071837 31868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:13.073104 31868 hybrid_clock.cc:648] HybridClock initialized: now 1786486693073057 us; error 58 us; skew 500 ppm
I20260811 22:18:13.073474 31868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:13 dist-test-slave-jq4t krb5kdc[31483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486693, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:13.076962 31868 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:18:13.078709 31868 webserver.cc:460] Webserver started at http://127.29.229.2:39003/ using document root <none> and password file <none>
I20260811 22:18:13.079515 31868 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:13.079677 31868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:13.080019 31868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:13.083258 31868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "70e08056a1f5453cabacf8a6687d9791"
format_stamp: "Formatted at 2026-08-11 22:18:13 on dist-test-slave-jq4t"
server_key: "73dea101b97f66c782aef99c79b97790"
server_key_iv: "09ffab37a522cce297220896e8794569"
server_key_version: "encryptionkey@0"
I20260811 22:18:13.084134 31868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "70e08056a1f5453cabacf8a6687d9791"
format_stamp: "Formatted at 2026-08-11 22:18:13 on dist-test-slave-jq4t"
server_key: "73dea101b97f66c782aef99c79b97790"
server_key_iv: "09ffab37a522cce297220896e8794569"
server_key_version: "encryptionkey@0"
I20260811 22:18:13.089597 31868 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 22:18:13.094686 31897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:13.097182 31868 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 22:18:13.097414 31868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "70e08056a1f5453cabacf8a6687d9791"
format_stamp: "Formatted at 2026-08-11 22:18:13 on dist-test-slave-jq4t"
server_key: "73dea101b97f66c782aef99c79b97790"
server_key_iv: "09ffab37a522cce297220896e8794569"
server_key_version: "encryptionkey@0"
I20260811 22:18:13.097906 31868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:13.113469 31868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:13.118413 31868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:13.118760 31868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:13.119880 31868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:13.121666 31868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:13.121778 31868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:13.122030 31868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:13.122167 31868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:13.163069 31868 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:43651
I20260811 22:18:13.163131 32081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:43651 every 8 connection(s)
I20260811 22:18:13.164393 31868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:18:13.172109 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 31868
I20260811 22:18:13.172343 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:18:13.172952 30612 external_mini_cluster.cc:1428] Setting key 59f48b2b93554ceda884d3b653935dba
Aug 11 22:18:13 dist-test-slave-jq4t krb5kdc[31483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486693, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:13.187194 32089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:13.167114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:58415 (local address 127.29.229.62:42131)
0811 22:18:13.167616 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:13.167625 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:13.168686 (+ 1061us) server_negotiation.cc:407] Connection header received
0811 22:18:13.170706 (+ 2020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:13.170711 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:13.170787 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:13.171031 (+ 244us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:13.172772 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.173757 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.175755 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.176219 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.176323 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:13.180984 (+ 4661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:13.180989 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:13.180992 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:13.181027 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:13.183931 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:13.184801 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:13.184805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:13.184808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:13.184941 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:13.185409 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:13.185413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:13.185415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:13.185680 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:13.185855 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:13.186573 (+ 718us) server_negotiation.cc:299] Negotiation successful
0811 22:18:13.186800 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":230,"threads_started":1}
I20260811 22:18:13.188216 32091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:13.167673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:42131 (local address 127.29.229.2:58415)
0811 22:18:13.168449 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:13.168556 (+ 107us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:13.170298 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:13.171231 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:13.171240 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:13.171812 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:13.172586 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:13.172609 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.174045 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.174051 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:13.175584 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:13.175607 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.176460 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.176470 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:13.176833 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:13.177677 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:13.177713 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:13.180723 (+ 3010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:13.184103 (+ 3380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:13.184113 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:13.184130 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:13.184660 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:13.185060 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:13.185065 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:13.185069 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:13.185267 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:13.185815 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:13.185825 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:13.186190 (+ 365us) client_negotiation.cc:769] Sending connection context
0811 22:18:13.186467 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 22:18:13.187050 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":303,"threads_started":1}
I20260811 22:18:13.190302 32084 heartbeater.cc:347] Connected to a master server at 127.29.229.62:42131
I20260811 22:18:13.190706 32084 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:13.191769 32084 heartbeater.cc:511] Master 127.29.229.62:42131 requested a full tablet report, sending...
I20260811 22:18:13.193722 31560 ts_manager.cc:194] Registered new tserver with Master: 70e08056a1f5453cabacf8a6687d9791 (127.29.229.2:43651)
I20260811 22:18:13.195010 31560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:58415
WARNING: no policy specified for kudu/127.29.229.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
I20260811 22:18:13.229012 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:42131
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:37139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:13.589413 32100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:13.589732 32100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:13.589915 32100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:13.597688 32100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:13.597826 32100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:13.598033 32100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:18:13.608529 32100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:37139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:42131
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:13.610001 32100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:13.611439 32100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:13.625955 32108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:13.625977 32109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:13.626161 32113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:13.627979 32100 server_base.cc:1034] running on GCE node
I20260811 22:18:13.628777 32100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:13.629861 32100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:13.631136 32100 hybrid_clock.cc:648] HybridClock initialized: now 1786486693631071 us; error 52 us; skew 500 ppm
I20260811 22:18:13.631567 32100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:13 dist-test-slave-jq4t krb5kdc[31483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486693, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:13.636185 32100 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:18:13.637996 32100 webserver.cc:460] Webserver started at http://127.29.229.3:46035/ using document root <none> and password file <none>
I20260811 22:18:13.638792 32100 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:13.638959 32100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:13.639317 32100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:13.642545 32100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "4e1802c42d254cd0be3a6576e96133df"
format_stamp: "Formatted at 2026-08-11 22:18:13 on dist-test-slave-jq4t"
server_key: "e05240dea0e10bf686a0f0cbac2d8101"
server_key_iv: "6758c616431e04b56367f907c1116608"
server_key_version: "encryptionkey@0"
I20260811 22:18:13.643513 32100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "4e1802c42d254cd0be3a6576e96133df"
format_stamp: "Formatted at 2026-08-11 22:18:13 on dist-test-slave-jq4t"
server_key: "e05240dea0e10bf686a0f0cbac2d8101"
server_key_iv: "6758c616431e04b56367f907c1116608"
server_key_version: "encryptionkey@0"
I20260811 22:18:13.645327 31858 heartbeater.cc:503] Master 127.29.229.62:42131 was elected leader, sending a full tablet report...
I20260811 22:18:13.650051 32100 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 22:18:13.654086 32124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:13.655807 32100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260811 22:18:13.656028 32100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "4e1802c42d254cd0be3a6576e96133df"
format_stamp: "Formatted at 2026-08-11 22:18:13 on dist-test-slave-jq4t"
server_key: "e05240dea0e10bf686a0f0cbac2d8101"
server_key_iv: "6758c616431e04b56367f907c1116608"
server_key_version: "encryptionkey@0"
I20260811 22:18:13.656270 32100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:13.681526 32100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:13.686618 32100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:13.686980 32100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:13.688117 32100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:13.689746 32100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:13.689862 32100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:13.690026 32100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:13.690114 32100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:13.729895 32100 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:36857
I20260811 22:18:13.729992 32326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:36857 every 8 connection(s)
I20260811 22:18:13.731319 32100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:18:13.738387 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 32100
I20260811 22:18:13.738605 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:18:13.739058 30612 external_mini_cluster.cc:1428] Setting key ca786af48acb21dcac8adae18607ab2b
Aug 11 22:18:13 dist-test-slave-jq4t krb5kdc[31483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486693, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:13.756565 32331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:13.734115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:47271 (local address 127.29.229.62:42131)
0811 22:18:13.734651 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:13.734655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:13.735556 (+ 901us) server_negotiation.cc:407] Connection header received
0811 22:18:13.737751 (+ 2195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:13.737757 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:13.737938 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:13.738082 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:13.740230 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.741477 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.743535 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.744076 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.744220 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:13.748900 (+ 4680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:13.748907 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:13.748910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:13.748961 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:13.752640 (+ 3679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:13.753856 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:13.753865 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:13.753868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:13.754041 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:13.754531 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:13.754539 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:13.754545 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:13.754873 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:13.755067 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:13.755839 (+ 772us) server_negotiation.cc:299] Negotiation successful
0811 22:18:13.756119 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":279,"threads_started":1}
I20260811 22:18:13.757704 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:13.734557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:42131 (local address 127.29.229.3:47271)
0811 22:18:13.735363 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:13.735412 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:13.737418 (+ 2006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:13.738365 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:13.738375 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:13.739148 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:13.739860 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:13.739893 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.741794 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.741801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:13.743363 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:13.743379 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.744333 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.744343 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:13.744664 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:13.745484 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:13.745510 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:13.748421 (+ 2911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:13.752902 (+ 4481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:13.752912 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:13.752930 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:13.753693 (+ 763us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:13.754169 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:13.754180 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:13.754183 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:13.754407 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:13.755014 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:13.755023 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:13.755432 (+ 409us) client_negotiation.cc:769] Sending connection context
0811 22:18:13.755736 (+ 304us) client_negotiation.cc:240] Negotiation successful
0811 22:18:13.756369 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":348,"threads_started":1}
I20260811 22:18:13.759856 32327 heartbeater.cc:347] Connected to a master server at 127.29.229.62:42131
I20260811 22:18:13.760362 32327 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:13.761758 32327 heartbeater.cc:511] Master 127.29.229.62:42131 requested a full tablet report, sending...
I20260811 22:18:13.763957 31560 ts_manager.cc:194] Registered new tserver with Master: 4e1802c42d254cd0be3a6576e96133df (127.29.229.3:36857)
I20260811 22:18:13.764968 31560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:47271
I20260811 22:18:13.768065 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:18:13.791760 32331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:13.775092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50528 (local address 127.29.229.62:42131)
0811 22:18:13.775356 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:13.775363 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:13.775715 (+ 352us) server_negotiation.cc:407] Connection header received
0811 22:18:13.775997 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:13.776002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:13.776143 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:13.776321 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:13.778164 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.779052 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.780580 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.781086 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.781251 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:13.784845 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:13.784850 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:13.784854 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:13.784900 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:13.787768 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:13.789319 (+ 1551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:13.789323 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:13.789326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:13.789481 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:13.790058 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:13.790062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:13.790064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:13.790473 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:13.790692 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:13.791225 (+ 533us) server_negotiation.cc:299] Negotiation successful
0811 22:18:13.791429 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:18:13.799266 31560 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50528:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:18:13.808943 31560 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:18:13.852442 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:13.823197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:36857 (local address 127.0.0.1:40228)
0811 22:18:13.833696 (+ 10499us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:13.833740 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:13.833974 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:13.842564 (+ 8590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:13.842574 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:13.842618 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:13.843229 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:13.843243 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.845901 (+ 2658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.845906 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:13.849013 (+ 3107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:13.849027 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.851287 (+ 2260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.851294 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:13.851516 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:13.851589 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 22:18:13.851710 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 22:18:13.851945 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10222,"mutex_wait_us":23,"thread_start_us":18892,"threads_started":1}
I20260811 22:18:13.852560 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:13.824976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45976 (local address 127.29.229.1:36757)
0811 22:18:13.844169 (+ 19193us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:13.844179 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:13.844211 (+ 32us) server_negotiation.cc:407] Connection header received
0811 22:18:13.844321 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:13.844329 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:13.844637 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:13.844869 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:13.845798 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.847368 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.849473 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.851039 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.851190 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:13.851461 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:13.852005 (+ 544us) server_negotiation.cc:299] Negotiation successful
0811 22:18:13.852194 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18949,"thread_start_us":18772,"threads_started":1}
I20260811 22:18:13.853363 32351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:13.823461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40228 (local address 127.29.229.3:36857)
0811 22:18:13.841668 (+ 18207us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:13.841683 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:13.841736 (+ 53us) server_negotiation.cc:407] Connection header received
0811 22:18:13.841892 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:13.841904 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:13.842271 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:13.843586 (+ 1315us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:13.843953 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.845717 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.849183 (+ 3466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.851039 (+ 1856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.852472 (+ 1433us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:13.852756 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:13.852826 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 22:18:13.852992 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17910,"thread_start_us":19233,"threads_started":1}
I20260811 22:18:13.853612 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:13.823968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:36757 (local address 127.0.0.1:45976)
0811 22:18:13.843302 (+ 19334us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:13.843322 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:13.843426 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:13.845020 (+ 1594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:13.845026 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:13.845054 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:13.845567 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:13.845586 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.847493 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.847500 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:13.849285 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:13.849299 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.851618 (+ 2319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.851624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:13.851797 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:13.851829 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:18:13.851916 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 22:18:13.852021 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19186,"mutex_wait_us":2,"thread_start_us":9276,"threads_started":1}
I20260811 22:18:13.853945 32349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:13.823537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:43651 (local address 127.0.0.1:33620)
0811 22:18:13.842439 (+ 18902us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:13.842460 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:13.842600 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:13.845936 (+ 3336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:13.845941 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:13.845978 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:13.846898 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:13.846914 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.848922 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.848927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:13.851167 (+ 2240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:13.851183 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.853062 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.853066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:13.853241 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:13.853272 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 22:18:13.853365 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 22:18:13.853531 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18742,"mutex_wait_us":40,"thread_start_us":19833,"threads_started":1}
I20260811 22:18:13.854014 32356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:13.824347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33620 (local address 127.29.229.2:43651)
0811 22:18:13.845035 (+ 20688us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:13.845043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:13.845069 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:18:13.845169 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:13.845178 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:13.845407 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:13.845660 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:13.847091 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.848724 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.851383 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:13.852677 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:13.852818 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:13.852948 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:13.853437 (+ 489us) server_negotiation.cc:299] Negotiation successful
0811 22:18:13.853642 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20523,"thread_start_us":20355,"threads_started":1}
I20260811 22:18:13.857522 31761 tablet_service.cc:1467] Processing CreateTablet for tablet a5be2b3e3d3e44df83187d59227508e3 (DEFAULT_TABLE table=test-table [id=9451e11ac798467596987dafcf3c9e0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:13.857720 31973 tablet_service.cc:1467] Processing CreateTablet for tablet a5be2b3e3d3e44df83187d59227508e3 (DEFAULT_TABLE table=test-table [id=9451e11ac798467596987dafcf3c9e0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:13.858302 32211 tablet_service.cc:1467] Processing CreateTablet for tablet a5be2b3e3d3e44df83187d59227508e3 (DEFAULT_TABLE table=test-table [id=9451e11ac798467596987dafcf3c9e0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:13.858934 31761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5be2b3e3d3e44df83187d59227508e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:13.859287 31973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5be2b3e3d3e44df83187d59227508e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:13.860654 32211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5be2b3e3d3e44df83187d59227508e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:13.871392 32361 tablet_bootstrap.cc:492] T a5be2b3e3d3e44df83187d59227508e3 P fb39f83ab39948848ca58f3b63bc27bf: Bootstrap starting.
I20260811 22:18:13.875118 32361 tablet_bootstrap.cc:654] T a5be2b3e3d3e44df83187d59227508e3 P fb39f83ab39948848ca58f3b63bc27bf: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:13.876209 32361 log.cc:824] T a5be2b3e3d3e44df83187d59227508e3 P fb39f83ab39948848ca58f3b63bc27bf: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:13.877372 32364 tablet_bootstrap.cc:492] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791: Bootstrap starting.
I20260811 22:18:13.878968 32371 tablet_bootstrap.cc:492] T a5be2b3e3d3e44df83187d59227508e3 P 4e1802c42d254cd0be3a6576e96133df: Bootstrap starting.
I20260811 22:18:13.881909 32364 tablet_bootstrap.cc:654] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:13.882622 32371 tablet_bootstrap.cc:654] T a5be2b3e3d3e44df83187d59227508e3 P 4e1802c42d254cd0be3a6576e96133df: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:13.883430 32364 log.cc:824] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:13.883630 32361 tablet_bootstrap.cc:492] T a5be2b3e3d3e44df83187d59227508e3 P fb39f83ab39948848ca58f3b63bc27bf: No bootstrap required, opened a new log
I20260811 22:18:13.883947 32361 ts_tablet_manager.cc:1397] T a5be2b3e3d3e44df83187d59227508e3 P fb39f83ab39948848ca58f3b63bc27bf: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20260811 22:18:13.883982 32371 log.cc:824] T a5be2b3e3d3e44df83187d59227508e3 P 4e1802c42d254cd0be3a6576e96133df: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:13.888631 32364 tablet_bootstrap.cc:492] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791: No bootstrap required, opened a new log
I20260811 22:18:13.889045 32364 ts_tablet_manager.cc:1397] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260811 22:18:13.891563 32371 tablet_bootstrap.cc:492] T a5be2b3e3d3e44df83187d59227508e3 P 4e1802c42d254cd0be3a6576e96133df: No bootstrap required, opened a new log
I20260811 22:18:13.891870 32371 ts_tablet_manager.cc:1397] T a5be2b3e3d3e44df83187d59227508e3 P 4e1802c42d254cd0be3a6576e96133df: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260811 22:18:13.895483 32361 raft_consensus.cc:348] T a5be2b3e3d3e44df83187d59227508e3 P fb39f83ab39948848ca58f3b63bc27bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1802c42d254cd0be3a6576e96133df" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36857 } } peers { permanent_uuid: "70e08056a1f5453cabacf8a6687d9791" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 43651 } } peers { permanent_uuid: "fb39f83ab39948848ca58f3b63bc27bf" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36757 } }
I20260811 22:18:13.895967 32361 raft_consensus.cc:374] T a5be2b3e3d3e44df83187d59227508e3 P fb39f83ab39948848ca58f3b63bc27bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:13.896126 32361 raft_consensus.cc:729] T a5be2b3e3d3e44df83187d59227508e3 P fb39f83ab39948848ca58f3b63bc27bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb39f83ab39948848ca58f3b63bc27bf, State: Initialized, Role: FOLLOWER
I20260811 22:18:13.896864 32361 consensus_queue.cc:260] T a5be2b3e3d3e44df83187d59227508e3 P fb39f83ab39948848ca58f3b63bc27bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1802c42d254cd0be3a6576e96133df" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36857 } } peers { permanent_uuid: "70e08056a1f5453cabacf8a6687d9791" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 43651 } } peers { permanent_uuid: "fb39f83ab39948848ca58f3b63bc27bf" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36757 } }
I20260811 22:18:13.898921 32361 ts_tablet_manager.cc:1428] T a5be2b3e3d3e44df83187d59227508e3 P fb39f83ab39948848ca58f3b63bc27bf: Time spent starting tablet: real 0.015s user 0.010s sys 0.003s
I20260811 22:18:13.900460 32364 raft_consensus.cc:348] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1802c42d254cd0be3a6576e96133df" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36857 } } peers { permanent_uuid: "70e08056a1f5453cabacf8a6687d9791" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 43651 } } peers { permanent_uuid: "fb39f83ab39948848ca58f3b63bc27bf" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36757 } }
I20260811 22:18:13.900946 32364 raft_consensus.cc:374] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:13.901065 32364 raft_consensus.cc:729] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70e08056a1f5453cabacf8a6687d9791, State: Initialized, Role: FOLLOWER
I20260811 22:18:13.901731 32364 consensus_queue.cc:260] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1802c42d254cd0be3a6576e96133df" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36857 } } peers { permanent_uuid: "70e08056a1f5453cabacf8a6687d9791" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 43651 } } peers { permanent_uuid: "fb39f83ab39948848ca58f3b63bc27bf" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36757 } }
I20260811 22:18:13.901640 32371 raft_consensus.cc:348] T a5be2b3e3d3e44df83187d59227508e3 P 4e1802c42d254cd0be3a6576e96133df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1802c42d254cd0be3a6576e96133df" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36857 } } peers { permanent_uuid: "70e08056a1f5453cabacf8a6687d9791" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 43651 } } peers { permanent_uuid: "fb39f83ab39948848ca58f3b63bc27bf" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36757 } }
I20260811 22:18:13.902132 32371 raft_consensus.cc:374] T a5be2b3e3d3e44df83187d59227508e3 P 4e1802c42d254cd0be3a6576e96133df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:13.902246 32371 raft_consensus.cc:729] T a5be2b3e3d3e44df83187d59227508e3 P 4e1802c42d254cd0be3a6576e96133df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e1802c42d254cd0be3a6576e96133df, State: Initialized, Role: FOLLOWER
I20260811 22:18:13.903107 32084 heartbeater.cc:503] Master 127.29.229.62:42131 was elected leader, sending a full tablet report...
I20260811 22:18:13.903038 32371 consensus_queue.cc:260] T a5be2b3e3d3e44df83187d59227508e3 P 4e1802c42d254cd0be3a6576e96133df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1802c42d254cd0be3a6576e96133df" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36857 } } peers { permanent_uuid: "70e08056a1f5453cabacf8a6687d9791" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 43651 } } peers { permanent_uuid: "fb39f83ab39948848ca58f3b63bc27bf" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36757 } }
I20260811 22:18:13.904516 32327 heartbeater.cc:503] Master 127.29.229.62:42131 was elected leader, sending a full tablet report...
I20260811 22:18:13.906426 32364 ts_tablet_manager.cc:1428] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791: Time spent starting tablet: real 0.017s user 0.007s sys 0.007s
I20260811 22:18:13.906934 32371 ts_tablet_manager.cc:1428] T a5be2b3e3d3e44df83187d59227508e3 P 4e1802c42d254cd0be3a6576e96133df: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
W20260811 22:18:13.916662 32086 tablet.cc:2369] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:18:13.983210 32328 tablet.cc:2369] T a5be2b3e3d3e44df83187d59227508e3 P 4e1802c42d254cd0be3a6576e96133df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:18:14.001260 32377 raft_consensus.cc:482] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:18:14.001598 32377 raft_consensus.cc:504] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1802c42d254cd0be3a6576e96133df" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36857 } } peers { permanent_uuid: "70e08056a1f5453cabacf8a6687d9791" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 43651 } } peers { permanent_uuid: "fb39f83ab39948848ca58f3b63bc27bf" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36757 } }
I20260811 22:18:14.003351 32377 leader_election.cc:288] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e1802c42d254cd0be3a6576e96133df (127.29.229.3:36857), fb39f83ab39948848ca58f3b63bc27bf (127.29.229.1:36757)
I20260811 22:18:14.012753 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:14.004563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:56831 (local address 127.29.229.1:36757)
0811 22:18:14.004784 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:14.004790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:14.005174 (+ 384us) server_negotiation.cc:407] Connection header received
0811 22:18:14.005350 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:14.005356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:14.005439 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:14.005526 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:14.006191 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.007325 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.010204 (+ 2879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.011495 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.011674 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:14.011744 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:14.012316 (+ 572us) server_negotiation.cc:299] Negotiation successful
0811 22:18:14.012459 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:18:14.012753 32351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:14.004438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:40679 (local address 127.29.229.3:36857)
0811 22:18:14.004705 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:14.004712 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:14.004739 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:18:14.005023 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:14.005029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:14.005116 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:14.005232 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:14.005916 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.007169 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.010195 (+ 3026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.011495 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.011665 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:14.011744 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:14.012316 (+ 572us) server_negotiation.cc:299] Negotiation successful
0811 22:18:14.012459 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260811 22:18:14.013000 32379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:14.003792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:36857 (local address 127.29.229.2:40679)
0811 22:18:14.004625 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:14.004650 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:14.004821 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:14.005263 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:14.005271 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:14.005297 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:14.005681 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:14.005694 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.007299 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.007303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:14.010012 (+ 2709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:14.010028 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.011807 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.011814 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:14.012080 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:14.012143 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 22:18:14.012246 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 22:18:14.012440 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"mutex_wait_us":138,"thread_start_us":383,"threads_started":1}
I20260811 22:18:14.013350 32380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:14.003838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:36757 (local address 127.29.229.2:56831)
0811 22:18:14.005071 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:14.005084 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:14.005202 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:14.005575 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:14.005582 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:14.005615 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:14.006019 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:14.006028 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.007471 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.007478 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:14.010012 (+ 2534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:14.010028 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.011803 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.011816 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:14.012084 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:14.012154 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 22:18:14.012246 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 22:18:14.012440 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1126,"mutex_wait_us":247,"thread_start_us":1047,"threads_started":1}
I20260811 22:18:14.014192 32247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5be2b3e3d3e44df83187d59227508e3" candidate_uuid: "70e08056a1f5453cabacf8a6687d9791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e1802c42d254cd0be3a6576e96133df" is_pre_election: true
I20260811 22:18:14.014485 31786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5be2b3e3d3e44df83187d59227508e3" candidate_uuid: "70e08056a1f5453cabacf8a6687d9791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb39f83ab39948848ca58f3b63bc27bf" is_pre_election: true
I20260811 22:18:14.014705 32247 raft_consensus.cc:2454] T a5be2b3e3d3e44df83187d59227508e3 P 4e1802c42d254cd0be3a6576e96133df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70e08056a1f5453cabacf8a6687d9791 in term 0.
I20260811 22:18:14.015020 31786 raft_consensus.cc:2454] T a5be2b3e3d3e44df83187d59227508e3 P fb39f83ab39948848ca58f3b63bc27bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70e08056a1f5453cabacf8a6687d9791 in term 0.
I20260811 22:18:14.015551 31900 leader_election.cc:302] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e1802c42d254cd0be3a6576e96133df, 70e08056a1f5453cabacf8a6687d9791; no voters:
I20260811 22:18:14.015969 32377 raft_consensus.cc:2781] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:18:14.016134 32377 raft_consensus.cc:482] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:18:14.016247 32377 raft_consensus.cc:3037] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:14.018605 32377 raft_consensus.cc:504] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1802c42d254cd0be3a6576e96133df" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36857 } } peers { permanent_uuid: "70e08056a1f5453cabacf8a6687d9791" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 43651 } } peers { permanent_uuid: "fb39f83ab39948848ca58f3b63bc27bf" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36757 } }
I20260811 22:18:14.019245 32377 leader_election.cc:288] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791 [CANDIDATE]: Term 1 election: Requested vote from peers 4e1802c42d254cd0be3a6576e96133df (127.29.229.3:36857), fb39f83ab39948848ca58f3b63bc27bf (127.29.229.1:36757)
I20260811 22:18:14.019778 31786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5be2b3e3d3e44df83187d59227508e3" candidate_uuid: "70e08056a1f5453cabacf8a6687d9791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb39f83ab39948848ca58f3b63bc27bf"
I20260811 22:18:14.019825 32247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5be2b3e3d3e44df83187d59227508e3" candidate_uuid: "70e08056a1f5453cabacf8a6687d9791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e1802c42d254cd0be3a6576e96133df"
I20260811 22:18:14.020009 31786 raft_consensus.cc:3037] T a5be2b3e3d3e44df83187d59227508e3 P fb39f83ab39948848ca58f3b63bc27bf [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:14.020057 32247 raft_consensus.cc:3037] T a5be2b3e3d3e44df83187d59227508e3 P 4e1802c42d254cd0be3a6576e96133df [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:14.022431 31786 raft_consensus.cc:2454] T a5be2b3e3d3e44df83187d59227508e3 P fb39f83ab39948848ca58f3b63bc27bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70e08056a1f5453cabacf8a6687d9791 in term 1.
I20260811 22:18:14.022557 32247 raft_consensus.cc:2454] T a5be2b3e3d3e44df83187d59227508e3 P 4e1802c42d254cd0be3a6576e96133df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70e08056a1f5453cabacf8a6687d9791 in term 1.
I20260811 22:18:14.023039 31900 leader_election.cc:302] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e1802c42d254cd0be3a6576e96133df, 70e08056a1f5453cabacf8a6687d9791; no voters:
I20260811 22:18:14.023454 32377 raft_consensus.cc:2781] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:14.024230 32377 raft_consensus.cc:686] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791 [term 1 LEADER]: Becoming Leader. State: Replica: 70e08056a1f5453cabacf8a6687d9791, State: Running, Role: LEADER
I20260811 22:18:14.025002 32377 consensus_queue.cc:237] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1802c42d254cd0be3a6576e96133df" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36857 } } peers { permanent_uuid: "70e08056a1f5453cabacf8a6687d9791" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 43651 } } peers { permanent_uuid: "fb39f83ab39948848ca58f3b63bc27bf" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36757 } }
I20260811 22:18:14.031015 31557 catalog_manager.cc:5495] T a5be2b3e3d3e44df83187d59227508e3 P 70e08056a1f5453cabacf8a6687d9791 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70e08056a1f5453cabacf8a6687d9791 (127.29.229.2). New cstate: current_term: 1 leader_uuid: "70e08056a1f5453cabacf8a6687d9791" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1802c42d254cd0be3a6576e96133df" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70e08056a1f5453cabacf8a6687d9791" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 43651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb39f83ab39948848ca58f3b63bc27bf" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36757 } health_report { overall_health: UNKNOWN } } }
Aug 11 22:18:14 dist-test-slave-jq4t krb5kdc[31483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:18:14.055609 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:14.039959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45988 (local address 127.29.229.1:36757)
0811 22:18:14.040147 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:14.040152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:14.040775 (+ 623us) server_negotiation.cc:407] Connection header received
0811 22:18:14.041003 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:14.041007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:14.041072 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:14.041172 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:14.042762 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.043615 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.044926 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.045377 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.045514 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:14.049713 (+ 4199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:14.049727 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:14.049744 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:14.049787 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:14.052884 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:14.053575 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:14.053582 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:14.053585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:14.053747 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:14.054208 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:14.054212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:14.054214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:14.054604 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:14.054866 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:14.055153 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 22:18:14.055326 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 22:18:14 dist-test-slave-jq4t krb5kdc[31483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486694, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260811 22:18:14.113569 31859 tablet.cc:2369] T a5be2b3e3d3e44df83187d59227508e3 P fb39f83ab39948848ca58f3b63bc27bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 22:18:14 dist-test-slave-jq4t krb5kdc[31483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486694, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:18:14.123621 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:14.106094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45992 (local address 127.29.229.1:36757)
0811 22:18:14.106470 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:14.106480 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:14.106817 (+ 337us) server_negotiation.cc:407] Connection header received
0811 22:18:14.107040 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:14.107050 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:14.107140 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:14.107250 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:14.108824 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.109592 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.111631 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.112345 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.112593 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:14.117619 (+ 5026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:14.117626 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:14.117629 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:14.117680 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:14.120813 (+ 3133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:14.121565 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:14.121572 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:14.121580 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:14.121726 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:14.122141 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:14.122146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:14.122149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:14.122443 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:14.122725 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:14.123007 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 22:18:14.123265 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185}
Aug 11 22:18:14 dist-test-slave-jq4t krb5kdc[31483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:18:14 dist-test-slave-jq4t krb5kdc[31483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:18:14.186993 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:14.170122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46000 (local address 127.29.229.1:36757)
0811 22:18:14.170493 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:14.170503 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:14.171246 (+ 743us) server_negotiation.cc:407] Connection header received
0811 22:18:14.171474 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:14.171479 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:14.171575 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:14.171672 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:14.173613 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.174333 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.176217 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.176749 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.176854 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:14.181525 (+ 4671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:14.181530 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:14.181534 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:14.181572 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:14.184323 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:14.185032 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:14.185036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:14.185038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:14.185168 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:14.185538 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:14.185543 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:14.185545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:14.185877 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:14.186096 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:14.186398 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 22:18:14.186593 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
W20260811 22:18:14.189095 31725 tablet_service.cc:2908] Rejecting scan request for tablet a5be2b3e3d3e44df83187d59227508e3: Uninitialized: clean time has not yet been initialized
I20260811 22:18:14.193038 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 31657
I20260811 22:18:14.209000 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:14.194075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46002 (local address 127.29.229.1:36757)
0811 22:18:14.194243 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:14.194247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:14.194585 (+ 338us) server_negotiation.cc:407] Connection header received
0811 22:18:14.194737 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:14.194741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:14.194814 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:14.194899 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:14.196266 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.197131 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.198623 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.199097 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.199221 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:14.202382 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:14.202387 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:14.202390 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:14.202428 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:14.205405 (+ 2977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:14.206464 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:14.206469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:14.206472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:14.206804 (+ 332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:14.207378 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:14.207382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:14.207384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:14.207742 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:14.208003 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:14.208442 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 22:18:14.208725 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:18:14.209718 31849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:14.302374 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 31657
I20260811 22:18:14.332535 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 31868
Aug 11 22:18:14 dist-test-slave-jq4t krb5kdc[31483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.2@KRBTEST.COM
I20260811 22:18:14.348410 32356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:14.333380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33630 (local address 127.29.229.2:43651)
0811 22:18:14.333575 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:14.333580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:14.333598 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:18:14.333714 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:14.333718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:14.333782 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:14.333867 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:14.335671 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.336456 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.337905 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.338328 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.338436 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:14.342540 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:14.342554 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:14.342570 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:14.342615 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:14.345560 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:14.346243 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:14.346250 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:14.346253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:14.346407 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:14.346847 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:14.346856 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:14.346859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:14.347221 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:14.347534 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:14.347822 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 22:18:14.348093 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:18:14.349205 32075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:14.443946 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 31868
I20260811 22:18:14.465536 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 32100
Aug 11 22:18:14 dist-test-slave-jq4t krb5kdc[31483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.3@KRBTEST.COM
I20260811 22:18:14.480716 32351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:14.466365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40238 (local address 127.29.229.3:36857)
0811 22:18:14.466546 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:14.466551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:14.466570 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:18:14.466702 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:14.466706 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:14.466771 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:14.466882 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:14.468380 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.469166 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.470475 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:14.470898 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:14.471054 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:14.475113 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:14.475125 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:14.475141 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:14.475177 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:14.477911 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:14.478570 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:14.478578 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:14.478580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:14.478712 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:14.479103 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:14.479109 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:14.479112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:14.479504 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:14.479800 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:14.480110 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 22:18:14.480353 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:18:14.481359 32316 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:14.574384 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 32100
I20260811 22:18:14.602504 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 31506
I20260811 22:18:14.603711 31610 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:14.725265 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 31506
2026-08-11T22:18:14Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4891 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:18:14 dist-test-slave-jq4t krb5kdc[32411](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:18:14 dist-test-slave-jq4t krb5kdc[32411](info): set up 2 sockets
Aug 11 22:18:14 dist-test-slave-jq4t krb5kdc[32411](info): commencing operation
krb5kdc: starting...
W20260811 22:18:16.114123 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.327s user 0.002s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:18:16 dist-test-slave-jq4t krb5kdc[32411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:18:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:18:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:18:16.368759 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:44989
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:39001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:44989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:16.728709 32441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:16.729048 32441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:16.729146 32441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:16.737311 32441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:16.737445 32441 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:18:16.737528 32441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:16.737635 32441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:16.737732 32441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:16.737843 32441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:18:16.749226 32441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:39001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:44989
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:44989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:16.750691 32441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:16.752167 32441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:16.765928 32447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:16.765956 32452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:16.766021 32448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:16.772442 32441 server_base.cc:1034] running on GCE node
I20260811 22:18:16.773710 32441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:16.775410 32441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:16.776788 32441 hybrid_clock.cc:648] HybridClock initialized: now 1786486696776699 us; error 81 us; skew 500 ppm
I20260811 22:18:16.777338 32441 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:16 dist-test-slave-jq4t krb5kdc[32411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486696, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:16.784108 32441 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:18:16.786007 32441 webserver.cc:460] Webserver started at http://127.29.229.62:42959/ using document root <none> and password file <none>
I20260811 22:18:16.786791 32441 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:16.786906 32441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:16.787206 32441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:16.790448 32441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "95e81b8a7ec9467bb3d3a9bc0c0f0031"
format_stamp: "Formatted at 2026-08-11 22:18:16 on dist-test-slave-jq4t"
server_key: "6b95a960ff035a558e3b0fd3d6a56827"
server_key_iv: "68d11d500773ab76a28926e062d55a68"
server_key_version: "encryptionkey@0"
I20260811 22:18:16.791260 32441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "95e81b8a7ec9467bb3d3a9bc0c0f0031"
format_stamp: "Formatted at 2026-08-11 22:18:16 on dist-test-slave-jq4t"
server_key: "6b95a960ff035a558e3b0fd3d6a56827"
server_key_iv: "68d11d500773ab76a28926e062d55a68"
server_key_version: "encryptionkey@0"
I20260811 22:18:16.796972 32441 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 22:18:16.800551 32465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:16.802026 32441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:18:16.802202 32441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "95e81b8a7ec9467bb3d3a9bc0c0f0031"
format_stamp: "Formatted at 2026-08-11 22:18:16 on dist-test-slave-jq4t"
server_key: "6b95a960ff035a558e3b0fd3d6a56827"
server_key_iv: "68d11d500773ab76a28926e062d55a68"
server_key_version: "encryptionkey@0"
I20260811 22:18:16.802413 32441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:16.818468 32441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:16.822975 32441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:16.823218 32441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:16.849625 32441 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:44989
I20260811 22:18:16.849675 32577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:44989 every 8 connection(s)
I20260811 22:18:16.851120 32441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:16.855141 32578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:16.858479 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 32441
I20260811 22:18:16.858752 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:18:16.859467 30612 external_mini_cluster.cc:1428] Setting key 41bf834ad529707fa41125f9fc8f420d
I20260811 22:18:16.871507 32578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031: Bootstrap starting.
I20260811 22:18:16.875885 32578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031: Neither blocks nor log segments found. Creating new log.
Aug 11 22:18:16 dist-test-slave-jq4t krb5kdc[32411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:16.877087 32578 log.cc:824] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:16.880931 32578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031: No bootstrap required, opened a new log
I20260811 22:18:16.889012 32584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:16.862295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33112 (local address 127.29.229.62:44989)
0811 22:18:16.863518 (+ 1223us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:16.863536 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:16.863600 (+ 64us) server_negotiation.cc:407] Connection header received
0811 22:18:16.865747 (+ 2147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:16.865784 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:16.866552 (+ 768us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:16.867245 (+ 693us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:16.868843 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:16.870809 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:16.872269 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:16.873527 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:16.873782 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:16.878130 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:16.878138 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:16.878158 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:16.878208 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:16.882385 (+ 4177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:16.883174 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:16.883183 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:16.883192 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:16.883431 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:16.884050 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:16.884061 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:16.884067 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:16.885179 (+ 1112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:16.885661 (+ 482us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:16.886101 (+ 440us) server_negotiation.cc:299] Negotiation successful
0811 22:18:16.886810 (+ 709us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":895,"thread_start_us":578,"threads_started":1}
I20260811 22:18:16.892916 32578 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95e81b8a7ec9467bb3d3a9bc0c0f0031" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44989 } }
I20260811 22:18:16.893301 32578 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:16.893378 32578 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95e81b8a7ec9467bb3d3a9bc0c0f0031, State: Initialized, Role: FOLLOWER
I20260811 22:18:16.893870 32578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95e81b8a7ec9467bb3d3a9bc0c0f0031" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44989 } }
I20260811 22:18:16.894161 32578 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:18:16.894305 32578 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:18:16.894531 32578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:16.897120 32578 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95e81b8a7ec9467bb3d3a9bc0c0f0031" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44989 } }
I20260811 22:18:16.897704 32578 leader_election.cc:302] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95e81b8a7ec9467bb3d3a9bc0c0f0031; no voters:
I20260811 22:18:16.898360 32578 leader_election.cc:288] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:18:16.898501 32586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:16.899839 32586 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031 [term 1 LEADER]: Becoming Leader. State: Replica: 95e81b8a7ec9467bb3d3a9bc0c0f0031, State: Running, Role: LEADER
I20260811 22:18:16.900389 32586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95e81b8a7ec9467bb3d3a9bc0c0f0031" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44989 } }
I20260811 22:18:16.901252 32578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:16.903713 32587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95e81b8a7ec9467bb3d3a9bc0c0f0031" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95e81b8a7ec9467bb3d3a9bc0c0f0031" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44989 } } }
I20260811 22:18:16.903704 32588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95e81b8a7ec9467bb3d3a9bc0c0f0031. Latest consensus state: current_term: 1 leader_uuid: "95e81b8a7ec9467bb3d3a9bc0c0f0031" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95e81b8a7ec9467bb3d3a9bc0c0f0031" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44989 } } }
I20260811 22:18:16.904156 32587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:16.904346 32588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:16.907145 32592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:16.912889 32592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:16.921821 32592 catalog_manager.cc:1347] Generated new cluster ID: 49761839a6e24d70a892f2bca2c93781
I20260811 22:18:16.921970 32592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:16.944840 32592 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:18:16.946226 32592 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:16.970448 32592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031: Generated new TSK 0
I20260811 22:18:16.971343 32592 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
I20260811 22:18:17.033874 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:44989
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:39001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:17.392560 32620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:17.392984 32620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:17.393147 32620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:17.402406 32620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:17.402616 32620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:17.402731 32620 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:18:17.402916 32620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:18:17.413897 32620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:39001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.229.62:44989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:17.415175 32620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:17.416613 32620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:17.430472 32626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:17.430588 32627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:17.430641 32630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:17.431883 32620 server_base.cc:1034] running on GCE node
I20260811 22:18:17.432739 32620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:17.433779 32620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:17.435050 32620 hybrid_clock.cc:648] HybridClock initialized: now 1786486697434968 us; error 51 us; skew 500 ppm
I20260811 22:18:17.435391 32620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:17 dist-test-slave-jq4t krb5kdc[32411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486697, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:17.438905 32620 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:18:17.440725 32620 webserver.cc:460] Webserver started at http://127.29.229.1:33063/ using document root <none> and password file <none>
I20260811 22:18:17.441530 32620 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:17.441663 32620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:17.442132 32620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:17.445381 32620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "139832b153ef410f9ec46ab7e2656413"
format_stamp: "Formatted at 2026-08-11 22:18:17 on dist-test-slave-jq4t"
server_key: "ebac7a775f56dd25250c1288406e1503"
server_key_iv: "e27a185c1271ba1792e89a3021138584"
server_key_version: "encryptionkey@0"
I20260811 22:18:17.446224 32620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "139832b153ef410f9ec46ab7e2656413"
format_stamp: "Formatted at 2026-08-11 22:18:17 on dist-test-slave-jq4t"
server_key: "ebac7a775f56dd25250c1288406e1503"
server_key_iv: "e27a185c1271ba1792e89a3021138584"
server_key_version: "encryptionkey@0"
I20260811 22:18:17.451717 32620 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 22:18:17.455642 32640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:17.457178 32620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:18:17.457358 32620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "139832b153ef410f9ec46ab7e2656413"
format_stamp: "Formatted at 2026-08-11 22:18:17 on dist-test-slave-jq4t"
server_key: "ebac7a775f56dd25250c1288406e1503"
server_key_iv: "e27a185c1271ba1792e89a3021138584"
server_key_version: "encryptionkey@0"
I20260811 22:18:17.457599 32620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:17.474992 32620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:17.479328 32620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:17.479715 32620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:17.482769 32620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:17.482914 32620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:17.483072 32620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:17.483148 32620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:18:17 dist-test-slave-jq4t krb5kdc[32411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486697, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:17.507735 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:17.483455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:43293 (local address 127.29.229.62:44989)
0811 22:18:17.484096 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:17.484105 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:17.485359 (+ 1254us) server_negotiation.cc:407] Connection header received
0811 22:18:17.487895 (+ 2536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:17.487903 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:17.488000 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:17.488092 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:17.490919 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:17.491967 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:17.494783 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:17.495449 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:17.495599 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:17.500559 (+ 4960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:17.500570 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:17.500577 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:17.500631 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:17.504355 (+ 3724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:17.505217 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:17.505226 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:17.505230 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:17.505372 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:17.505858 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:17.505862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:17.505866 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:17.506146 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:17.506328 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:17.507100 (+ 772us) server_negotiation.cc:299] Negotiation successful
0811 22:18:17.507337 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":371,"threads_started":1}
I20260811 22:18:17.509063 32664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:17.484159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:44989 (local address 127.29.229.1:43293)
0811 22:18:17.485168 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:17.485229 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:17.487466 (+ 2237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:17.488766 (+ 1300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:17.488781 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:17.489619 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:17.490691 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:17.490729 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:17.492154 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:17.492158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:17.494616 (+ 2458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:17.494630 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:17.495712 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:17.495722 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:17.495955 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:17.496830 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:17.496870 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:17.500251 (+ 3381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:17.504532 (+ 4281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:17.504541 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:17.504562 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:17.505081 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:17.505486 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:17.505491 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:17.505495 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:17.505722 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:17.506275 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:17.506284 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:17.506692 (+ 408us) client_negotiation.cc:769] Sending connection context
0811 22:18:17.506975 (+ 283us) client_negotiation.cc:240] Negotiation successful
0811 22:18:17.507565 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"thread_start_us":414,"threads_started":1}
I20260811 22:18:17.536737 32620 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:38075
I20260811 22:18:17.536906 310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:17.518444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:44989 (local address 127.29.229.1:48197)
0811 22:18:17.519371 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:17.519398 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:17.519607 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:17.520046 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:17.520050 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:17.520593 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:17.521137 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:17.521160 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:17.523176 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:17.523183 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:17.524430 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:17.524445 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:17.525653 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:17.525661 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:17.525881 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:17.526603 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:17.526622 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:17.528021 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:17.532795 (+ 4774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:17.532804 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:17.532812 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:17.533709 (+ 897us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:17.534566 (+ 857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:17.534571 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:17.534574 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:17.534750 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:17.535304 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:17.535312 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:17.535592 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 22:18:17.535801 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 22:18:17.536264 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":548,"threads_started":1}
I20260811 22:18:17.537300 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:17.518416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:48197 (local address 127.29.229.62:44989)
0811 22:18:17.518613 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:17.518621 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:17.519539 (+ 918us) server_negotiation.cc:407] Connection header received
0811 22:18:17.519773 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:17.519780 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:17.519885 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:17.520014 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:17.521330 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:17.522976 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:17.524644 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:17.525420 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:17.525584 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:17.528276 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:17.528285 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:17.528290 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:17.528327 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:17.532620 (+ 4293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:17.534030 (+ 1410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:17.534040 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:17.534045 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:17.534405 (+ 360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:17.534873 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:17.534878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:17.534880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:17.535169 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:17.535325 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:17.536733 (+ 1408us) server_negotiation.cc:299] Negotiation successful
0811 22:18:17.537017 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:18:17.536767 393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:38075 every 8 connection(s)
I20260811 22:18:17.538431 32620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:18:17.541203 394 heartbeater.cc:347] Connected to a master server at 127.29.229.62:44989
I20260811 22:18:17.541496 394 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:17.542313 394 heartbeater.cc:511] Master 127.29.229.62:44989 requested a full tablet report, sending...
I20260811 22:18:17.544298 32503 ts_manager.cc:194] Registered new tserver with Master: 139832b153ef410f9ec46ab7e2656413 (127.29.229.1:38075)
I20260811 22:18:17.545815 32503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:43293
I20260811 22:18:17.548751 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 32620
I20260811 22:18:17.548977 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:18:17.549633 30612 external_mini_cluster.cc:1428] Setting key c186505d757cf70f0f2638a26a443f29
WARNING: no policy specified for kudu/127.29.229.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
I20260811 22:18:17.602134 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:44989
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:39001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:17.965889 402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:17.966185 402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:17.966357 402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:17.973871 402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:17.973995 402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:17.974144 402 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:18:17.974267 402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:18:17.984536 402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:39001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.229.62:44989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:17.986024 402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:17.987491 402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:18.001669 407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:18.002032 413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:18.001722 410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:18.002985 402 server_base.cc:1034] running on GCE node
I20260811 22:18:18.003682 402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:18.004837 402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:18.006242 402 hybrid_clock.cc:648] HybridClock initialized: now 1786486698006162 us; error 114 us; skew 500 ppm
I20260811 22:18:18.006589 402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:18 dist-test-slave-jq4t krb5kdc[32411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486698, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:18.010578 402 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:18:18.012593 402 webserver.cc:460] Webserver started at http://127.29.229.2:37939/ using document root <none> and password file <none>
I20260811 22:18:18.013377 402 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:18.013520 402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:18.013880 402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:18.017107 402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "0080c63ee5974e5d9f2ffe86a1876f62"
format_stamp: "Formatted at 2026-08-11 22:18:18 on dist-test-slave-jq4t"
server_key: "05b84ebe3b2bd387c3fefaa400436bc5"
server_key_iv: "034fa7447bbcccd64a15d629524ffd72"
server_key_version: "encryptionkey@0"
I20260811 22:18:18.017987 402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "0080c63ee5974e5d9f2ffe86a1876f62"
format_stamp: "Formatted at 2026-08-11 22:18:18 on dist-test-slave-jq4t"
server_key: "05b84ebe3b2bd387c3fefaa400436bc5"
server_key_iv: "034fa7447bbcccd64a15d629524ffd72"
server_key_version: "encryptionkey@0"
I20260811 22:18:18.023613 402 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:18:18.028303 421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:18.030615 402 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:18:18.030916 402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "0080c63ee5974e5d9f2ffe86a1876f62"
format_stamp: "Formatted at 2026-08-11 22:18:18 on dist-test-slave-jq4t"
server_key: "05b84ebe3b2bd387c3fefaa400436bc5"
server_key_iv: "034fa7447bbcccd64a15d629524ffd72"
server_key_version: "encryptionkey@0"
I20260811 22:18:18.031229 402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:18.042681 402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:18.047281 402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:18.047585 402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:18.050596 402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:18.050709 402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:18.050907 402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:18.050976 402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:18:18 dist-test-slave-jq4t krb5kdc[32411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486698, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:18.075393 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:18.051065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:54481 (local address 127.29.229.62:44989)
0811 22:18:18.051979 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:18.051983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:18.052777 (+ 794us) server_negotiation.cc:407] Connection header received
0811 22:18:18.054903 (+ 2126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:18.054909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:18.055086 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:18.055226 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:18.057783 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.058874 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.061744 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.062398 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.062540 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:18.067147 (+ 4607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:18.067152 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:18.067156 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:18.067190 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:18.071280 (+ 4090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:18.072080 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:18.072086 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:18.072090 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:18.072218 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:18.072754 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:18.072759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:18.072763 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:18.073217 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:18.073441 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:18.074478 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0811 22:18:18.074807 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":809,"thread_start_us":695,"threads_started":1}
I20260811 22:18:18.076141 441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:18.051745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:44989 (local address 127.29.229.2:54481)
0811 22:18:18.052629 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:18.052674 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:18.054604 (+ 1930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:18.055662 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:18.055674 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:18.056642 (+ 968us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:18.057574 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:18.057597 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.059087 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.059091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:18.061531 (+ 2440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:18.061549 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.062669 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.062676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:18.062934 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:18.063933 (+ 999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:18.063961 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:18.066905 (+ 2944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:18.071447 (+ 4542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:18.071457 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:18.071480 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:18.071940 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:18.072333 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:18.072338 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:18.072342 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:18.072629 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:18.073424 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:18.073433 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:18.073882 (+ 449us) client_negotiation.cc:769] Sending connection context
0811 22:18:18.074288 (+ 406us) client_negotiation.cc:240] Negotiation successful
0811 22:18:18.074868 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":366,"threads_started":1}
I20260811 22:18:18.100041 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:18.082861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:55451 (local address 127.29.229.62:44989)
0811 22:18:18.083084 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:18.083093 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:18.084090 (+ 997us) server_negotiation.cc:407] Connection header received
0811 22:18:18.084349 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:18.084353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:18.084423 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:18.084602 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:18.086264 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.087302 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.089473 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.090061 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.090137 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:18.093056 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:18.093061 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:18.093064 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:18.093096 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:18.096469 (+ 3373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:18.097556 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:18.097565 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:18.097570 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:18.097767 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:18.098280 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:18.098289 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:18.098293 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:18.098803 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:18.098990 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:18.099413 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 22:18:18.099672 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:18:18.100152 537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:18.082872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:44989 (local address 127.29.229.2:55451)
0811 22:18:18.083958 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:18.083984 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:18.084179 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:18.084645 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:18.084650 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:18.085434 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:18.086018 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:18.086043 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.087479 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.087486 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:18.089266 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:18.089285 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.090466 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.090475 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:18.090716 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:18.091537 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:18.091558 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:18.092839 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:18.096672 (+ 3833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:18.096678 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:18.096685 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:18.097392 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:18.097952 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:18.097959 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:18.097968 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:18.098138 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:18.098945 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:18.098950 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:18.099158 (+ 208us) client_negotiation.cc:769] Sending connection context
0811 22:18:18.099311 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 22:18:18.099677 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"thread_start_us":717,"threads_started":1}
I20260811 22:18:18.109115 402 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:44635
I20260811 22:18:18.109158 633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:44635 every 8 connection(s)
I20260811 22:18:18.110389 402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:18:18.112665 635 heartbeater.cc:347] Connected to a master server at 127.29.229.62:44989
I20260811 22:18:18.113001 635 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:18.113783 635 heartbeater.cc:511] Master 127.29.229.62:44989 requested a full tablet report, sending...
I20260811 22:18:18.115660 32503 ts_manager.cc:194] Registered new tserver with Master: 0080c63ee5974e5d9f2ffe86a1876f62 (127.29.229.2:44635)
I20260811 22:18:18.116798 32503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:54481
I20260811 22:18:18.117902 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 402
I20260811 22:18:18.118180 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:18:18.119072 30612 external_mini_cluster.cc:1428] Setting key 2f9264941101f9ade9d4d08e2a6941ef
WARNING: no policy specified for kudu/127.29.229.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
I20260811 22:18:18.173821 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:44989
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:39001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:18.533120 642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:18.533471 642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:18.533653 642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:18.541659 642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:18.541828 642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:18.541978 642 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:18:18.542120 642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:18:18.548655 394 heartbeater.cc:503] Master 127.29.229.62:44989 was elected leader, sending a full tablet report...
I20260811 22:18:18.552675 642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:39001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.229.62:44989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:18.553864 642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:18.555140 642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:18.568444 652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:18.568575 649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:18.568552 648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:18.569243 642 server_base.cc:1034] running on GCE node
I20260811 22:18:18.569860 642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:18.570776 642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:18.572063 642 hybrid_clock.cc:648] HybridClock initialized: now 1786486698571997 us; error 37 us; skew 500 ppm
I20260811 22:18:18.572422 642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:18 dist-test-slave-jq4t krb5kdc[32411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486698, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:18.576072 642 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:18:18.577797 642 webserver.cc:460] Webserver started at http://127.29.229.3:45109/ using document root <none> and password file <none>
I20260811 22:18:18.578583 642 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:18.578727 642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:18.579118 642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:18.582289 642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "c7b2dffc3ec34b289e979ca7f90242a8"
format_stamp: "Formatted at 2026-08-11 22:18:18 on dist-test-slave-jq4t"
server_key: "0df76d350f1d9eab12e48a9b89f5972d"
server_key_iv: "17e0c163d7e4b2da93cf17cc8a03538b"
server_key_version: "encryptionkey@0"
I20260811 22:18:18.583093 642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "c7b2dffc3ec34b289e979ca7f90242a8"
format_stamp: "Formatted at 2026-08-11 22:18:18 on dist-test-slave-jq4t"
server_key: "0df76d350f1d9eab12e48a9b89f5972d"
server_key_iv: "17e0c163d7e4b2da93cf17cc8a03538b"
server_key_version: "encryptionkey@0"
I20260811 22:18:18.589155 642 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 22:18:18.593276 664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:18.595415 642 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:18:18.595578 642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "c7b2dffc3ec34b289e979ca7f90242a8"
format_stamp: "Formatted at 2026-08-11 22:18:18 on dist-test-slave-jq4t"
server_key: "0df76d350f1d9eab12e48a9b89f5972d"
server_key_iv: "17e0c163d7e4b2da93cf17cc8a03538b"
server_key_version: "encryptionkey@0"
I20260811 22:18:18.595796 642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:18.639532 642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:18.644134 642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:18.644398 642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:18.647567 642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:18.647655 642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:18.647843 642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:18.647902 642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:18:18 dist-test-slave-jq4t krb5kdc[32411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486698, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:18.675238 675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:18.648743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:34527 (local address 127.29.229.62:44989)
0811 22:18:18.650960 (+ 2217us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:18.650966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:18.650988 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:18:18.652847 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:18.652853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:18.652971 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:18.653095 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:18.655431 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.657067 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.659680 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.660570 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.660726 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:18.666680 (+ 5954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:18.666691 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:18.666699 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:18.666757 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:18.671015 (+ 4258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:18.672325 (+ 1310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:18.672331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:18.672335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:18.672494 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:18.673100 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:18.673108 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:18.673113 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:18.673504 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:18.673656 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:18.674569 (+ 913us) server_negotiation.cc:299] Negotiation successful
0811 22:18:18.674811 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2077,"thread_start_us":1949,"threads_started":1}
I20260811 22:18:18.676309 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:18.649203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:44989 (local address 127.29.229.3:34527)
0811 22:18:18.650100 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:18.650151 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:18.652530 (+ 2379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:18.653411 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:18.653425 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:18.654315 (+ 890us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:18.655185 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:18.655236 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.657264 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.657270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:18.659518 (+ 2248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:18.659537 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.660911 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.660922 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:18.661281 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:18.662275 (+ 994us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:18.662305 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:18.666300 (+ 3995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:18.671237 (+ 4937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:18.671251 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:18.671276 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:18.672170 (+ 894us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:18.672670 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:18.672674 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:18.672677 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:18.672976 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:18.673755 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:18.673762 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:18.674181 (+ 419us) client_negotiation.cc:769] Sending connection context
0811 22:18:18.674467 (+ 286us) client_negotiation.cc:240] Negotiation successful
0811 22:18:18.675070 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":409,"threads_started":1}
I20260811 22:18:18.699753 801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:18.682822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:44989 (local address 127.29.229.3:53119)
0811 22:18:18.684270 (+ 1448us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:18.684289 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:18.684425 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:18.684923 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:18.684927 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:18.685426 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:18.685736 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:18.685746 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.687594 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.687599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:18.688765 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:18.688782 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.689747 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.689754 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:18.689909 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:18.690712 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:18.690744 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:18.692058 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:18.695852 (+ 3794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:18.695857 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:18.695861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:18.696739 (+ 878us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:18.697248 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:18.697254 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:18.697259 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:18.697463 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:18.698155 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:18.698160 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:18.698436 (+ 276us) client_negotiation.cc:769] Sending connection context
0811 22:18:18.698641 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 22:18:18.699044 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1277,"thread_start_us":1004,"threads_started":1}
I20260811 22:18:18.700770 675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:18.682957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:53119 (local address 127.29.229.62:44989)
0811 22:18:18.683321 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:18.683326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:18.684396 (+ 1070us) server_negotiation.cc:407] Connection header received
0811 22:18:18.684593 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:18.684602 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:18.684716 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:18.684864 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:18.685893 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.687402 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.688935 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.689531 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.689675 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:18.692314 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:18.692319 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:18.692323 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:18.692353 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:18.695717 (+ 3364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:18.696897 (+ 1180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:18.696905 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:18.696908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:18.697096 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:18.697614 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:18.697620 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:18.697623 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:18.698041 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:18.698205 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:18.700184 (+ 1979us) server_negotiation.cc:299] Negotiation successful
0811 22:18:18.700422 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221}
I20260811 22:18:18.707572 642 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:44875
I20260811 22:18:18.707624 881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:44875 every 8 connection(s)
I20260811 22:18:18.708941 642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:18:18.711205 883 heartbeater.cc:347] Connected to a master server at 127.29.229.62:44989
I20260811 22:18:18.711462 883 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:18.712203 883 heartbeater.cc:511] Master 127.29.229.62:44989 requested a full tablet report, sending...
I20260811 22:18:18.714056 32503 ts_manager.cc:194] Registered new tserver with Master: c7b2dffc3ec34b289e979ca7f90242a8 (127.29.229.3:44875)
I20260811 22:18:18.714277 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 642
I20260811 22:18:18.714527 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:18:18.715059 30612 external_mini_cluster.cc:1428] Setting key 27dd471f2537b48138cea0b1a3dfbd07
I20260811 22:18:18.715027 32503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:34527
I20260811 22:18:18.718838 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:18:18 dist-test-slave-jq4t krb5kdc[32411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486698, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:18:18 dist-test-slave-jq4t krb5kdc[32411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486698, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:18.780882 675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:18.765360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33120 (local address 127.29.229.62:44989)
0811 22:18:18.765622 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:18.765630 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:18.765873 (+ 243us) server_negotiation.cc:407] Connection header received
0811 22:18:18.766143 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:18.766148 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:18.766216 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:18.766324 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:18.767793 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.768607 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.769891 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.770324 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.770471 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:18.774878 (+ 4407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:18.774884 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:18.774887 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:18.774924 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:18.778142 (+ 3218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:18.778845 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:18.778851 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:18.778855 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:18.779023 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:18.779409 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:18.779415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:18.779418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:18.779699 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:18.779984 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:18.780264 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 22:18:18.780539 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 22:18:18.785868 32503 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:18:18.793179 32503 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:18:18.821610 906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:18.807936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:44875 (local address 127.0.0.1:44388)
0811 22:18:18.809090 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:18.809145 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:18.809343 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:18.810122 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:18.810129 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:18.810169 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:18.810554 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:18.810570 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.814451 (+ 3881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.814460 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:18.817710 (+ 3250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:18.817734 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.820205 (+ 2471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.820214 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:18.820563 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:18.820650 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 22:18:18.820792 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 22:18:18.821072 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":859,"thread_start_us":706,"threads_started":1}
I20260811 22:18:18.823194 907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:18.808379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44388 (local address 127.29.229.3:44875)
0811 22:18:18.809384 (+ 1005us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:18.809394 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:18.809429 (+ 35us) server_negotiation.cc:407] Connection header received
0811 22:18:18.809570 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:18.809577 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:18.809856 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:18.810766 (+ 910us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:18.811806 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.814246 (+ 2440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.818048 (+ 3802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.819874 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.822003 (+ 2129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:18.822344 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:18.822437 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 22:18:18.822687 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":821,"thread_start_us":609,"threads_started":1}
I20260811 22:18:18.827488 747 tablet_service.cc:1467] Processing CreateTablet for tablet c2afc468ea084c4a88dc101298b2f4e6 (DEFAULT_TABLE table=test-table [id=670522ebd9c040c0aed0cd29c5442db3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:18.829206 747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2afc468ea084c4a88dc101298b2f4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:18.833267 908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:18.809490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:44635 (local address 127.0.0.1:32978)
0811 22:18:18.810062 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:18.810081 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:18.810190 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:18.825772 (+ 15582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:18.825777 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:18.825809 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:18.826188 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:18.826201 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.827999 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.828005 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:18.830506 (+ 2501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:18.830522 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.832219 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.832226 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:18.832410 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:18.832445 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 22:18:18.832592 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 22:18:18.832770 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"spinlock_wait_cycles":22784,"thread_start_us":287,"threads_started":1}
I20260811 22:18:18.834091 909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:18.809787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32978 (local address 127.29.229.2:44635)
0811 22:18:18.824985 (+ 15198us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:18.824995 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:18.825028 (+ 33us) server_negotiation.cc:407] Connection header received
0811 22:18:18.825176 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:18.825188 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:18.825511 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:18.825805 (+ 294us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:18.826406 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.827842 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.830710 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.831994 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.832154 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:18.832365 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:18.833555 (+ 1190us) server_negotiation.cc:299] Negotiation successful
0811 22:18:18.833752 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15012,"thread_start_us":14872,"threads_started":1}
I20260811 22:18:18.834689 912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:18.810184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:38075 (local address 127.0.0.1:37780)
0811 22:18:18.820069 (+ 9885us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:18.820094 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:18.820268 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:18.826922 (+ 6654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:18.826927 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:18.826954 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:18.827361 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:18.827373 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.829351 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.829357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:18.831474 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:18.831489 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.833689 (+ 2200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.833696 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:18.833889 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:18.833925 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:18:18.834025 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 22:18:18.834366 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9684,"thread_start_us":10212,"threads_started":1}
I20260811 22:18:18.835242 914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:18.810746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37780 (local address 127.29.229.1:38075)
0811 22:18:18.826474 (+ 15728us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:18.826480 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:18.826503 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:18:18.826577 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:18.826582 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:18.826732 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:18.826936 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:18.827578 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.829138 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.831674 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.833185 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.833320 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:18.834689 (+ 1369us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:18.834772 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 22:18:18.834963 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15569,"thread_start_us":15432,"threads_started":1}
I20260811 22:18:18.838543 509 tablet_service.cc:1467] Processing CreateTablet for tablet c2afc468ea084c4a88dc101298b2f4e6 (DEFAULT_TABLE table=test-table [id=670522ebd9c040c0aed0cd29c5442db3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:18.838833 32733 tablet_service.cc:1467] Processing CreateTablet for tablet c2afc468ea084c4a88dc101298b2f4e6 (DEFAULT_TABLE table=test-table [id=670522ebd9c040c0aed0cd29c5442db3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:18.840232 32733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2afc468ea084c4a88dc101298b2f4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:18.841252 509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2afc468ea084c4a88dc101298b2f4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:18.844515 919 tablet_bootstrap.cc:492] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8: Bootstrap starting.
I20260811 22:18:18.849840 919 tablet_bootstrap.cc:654] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:18.851293 919 log.cc:824] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:18.858628 929 tablet_bootstrap.cc:492] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413: Bootstrap starting.
I20260811 22:18:18.862983 919 tablet_bootstrap.cc:492] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8: No bootstrap required, opened a new log
I20260811 22:18:18.863057 922 tablet_bootstrap.cc:492] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62: Bootstrap starting.
I20260811 22:18:18.863441 919 ts_tablet_manager.cc:1397] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.010s
I20260811 22:18:18.865252 929 tablet_bootstrap.cc:654] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:18.866505 929 log.cc:824] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:18.866612 922 tablet_bootstrap.cc:654] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:18.868527 922 log.cc:824] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:18.874861 922 tablet_bootstrap.cc:492] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62: No bootstrap required, opened a new log
I20260811 22:18:18.875169 922 ts_tablet_manager.cc:1397] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260811 22:18:18.877806 919 raft_consensus.cc:348] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } }
I20260811 22:18:18.878427 919 raft_consensus.cc:374] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:18.878551 919 raft_consensus.cc:729] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7b2dffc3ec34b289e979ca7f90242a8, State: Initialized, Role: FOLLOWER
I20260811 22:18:18.879480 919 consensus_queue.cc:260] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } }
I20260811 22:18:18.881242 883 heartbeater.cc:503] Master 127.29.229.62:44989 was elected leader, sending a full tablet report...
I20260811 22:18:18.881434 929 tablet_bootstrap.cc:492] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413: No bootstrap required, opened a new log
I20260811 22:18:18.881734 929 ts_tablet_manager.cc:1397] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413: Time spent bootstrapping tablet: real 0.023s user 0.007s sys 0.012s
I20260811 22:18:18.882072 919 ts_tablet_manager.cc:1428] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8: Time spent starting tablet: real 0.018s user 0.011s sys 0.004s
I20260811 22:18:18.885566 922 raft_consensus.cc:348] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } }
I20260811 22:18:18.886056 922 raft_consensus.cc:374] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:18.886175 922 raft_consensus.cc:729] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0080c63ee5974e5d9f2ffe86a1876f62, State: Initialized, Role: FOLLOWER
I20260811 22:18:18.886821 922 consensus_queue.cc:260] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } }
I20260811 22:18:18.888005 635 heartbeater.cc:503] Master 127.29.229.62:44989 was elected leader, sending a full tablet report...
I20260811 22:18:18.890059 922 ts_tablet_manager.cc:1428] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260811 22:18:18.891812 929 raft_consensus.cc:348] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } }
I20260811 22:18:18.892177 929 raft_consensus.cc:374] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:18.892292 929 raft_consensus.cc:729] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 139832b153ef410f9ec46ab7e2656413, State: Initialized, Role: FOLLOWER
I20260811 22:18:18.892881 929 consensus_queue.cc:260] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } }
I20260811 22:18:18.894408 929 ts_tablet_manager.cc:1428] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413: Time spent starting tablet: real 0.013s user 0.011s sys 0.002s
I20260811 22:18:18.946851 936 raft_consensus.cc:482] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:18:18.947239 936 raft_consensus.cc:504] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } }
I20260811 22:18:18.949002 936 leader_election.cc:288] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0080c63ee5974e5d9f2ffe86a1876f62 (127.29.229.2:44635), 139832b153ef410f9ec46ab7e2656413 (127.29.229.1:38075)
I20260811 22:18:18.955856 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:18.949471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:44635 (local address 127.29.229.3:46937)
0811 22:18:18.949702 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:18.949725 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:18.949904 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:18.950516 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:18.950521 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:18.950549 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:18.950815 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:18.950830 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.952067 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.952071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:18.954004 (+ 1933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:18.954020 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.955305 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.955309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:18.955453 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:18.955477 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:18:18.955552 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 22:18:18.955643 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":1}
I20260811 22:18:18.956043 909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:18.949616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:46937 (local address 127.29.229.2:44635)
0811 22:18:18.949846 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:18.949853 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:18.949876 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:18:18.950080 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:18.950089 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:18.950183 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:18.950311 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:18.950998 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.951917 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.954172 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.955152 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.955325 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:18.955405 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:18.955605 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 22:18:18.955749 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:18:18.957060 914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:18.950151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:51227 (local address 127.29.229.1:38075)
0811 22:18:18.950399 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:18.950406 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:18.950809 (+ 403us) server_negotiation.cc:407] Connection header received
0811 22:18:18.951000 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:18.951005 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:18.951106 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:18.951205 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:18.951776 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.952633 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.954728 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.955941 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.956117 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:18.956197 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:18.956663 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 22:18:18.956795 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 22:18:18.957099 941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:18.949902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:38075 (local address 127.29.229.3:51227)
0811 22:18:18.950675 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:18.950694 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:18.950852 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:18.951252 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:18.951256 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:18.951280 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:18.951597 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:18.951604 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.952767 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.952775 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:18.954577 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:18.954588 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:18.956299 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:18.956307 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:18.956453 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:18.956521 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 22:18:18.956609 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 22:18:18.956755 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":425,"threads_started":1}
I20260811 22:18:18.957247 556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2afc468ea084c4a88dc101298b2f4e6" candidate_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" is_pre_election: true
I20260811 22:18:18.957909 556 raft_consensus.cc:2454] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7b2dffc3ec34b289e979ca7f90242a8 in term 0.
I20260811 22:18:18.958037 301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2afc468ea084c4a88dc101298b2f4e6" candidate_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139832b153ef410f9ec46ab7e2656413" is_pre_election: true
I20260811 22:18:18.958640 301 raft_consensus.cc:2454] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7b2dffc3ec34b289e979ca7f90242a8 in term 0.
I20260811 22:18:18.958630 668 leader_election.cc:302] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0080c63ee5974e5d9f2ffe86a1876f62, c7b2dffc3ec34b289e979ca7f90242a8; no voters:
I20260811 22:18:18.959100 936 raft_consensus.cc:2781] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:18:18.959301 936 raft_consensus.cc:482] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:18:18.959415 936 raft_consensus.cc:3037] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Advancing to term 1
W20260811 22:18:18.960691 884 tablet.cc:2369] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:18:18.962631 936 raft_consensus.cc:504] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } }
I20260811 22:18:18.963402 936 leader_election.cc:288] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [CANDIDATE]: Term 1 election: Requested vote from peers 0080c63ee5974e5d9f2ffe86a1876f62 (127.29.229.2:44635), 139832b153ef410f9ec46ab7e2656413 (127.29.229.1:38075)
I20260811 22:18:18.964080 556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2afc468ea084c4a88dc101298b2f4e6" candidate_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0080c63ee5974e5d9f2ffe86a1876f62"
I20260811 22:18:18.964151 301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2afc468ea084c4a88dc101298b2f4e6" candidate_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139832b153ef410f9ec46ab7e2656413"
I20260811 22:18:18.964345 556 raft_consensus.cc:3037] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:18.964356 301 raft_consensus.cc:3037] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:18.967880 556 raft_consensus.cc:2454] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7b2dffc3ec34b289e979ca7f90242a8 in term 1.
I20260811 22:18:18.967880 301 raft_consensus.cc:2454] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7b2dffc3ec34b289e979ca7f90242a8 in term 1.
I20260811 22:18:18.968361 668 leader_election.cc:302] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139832b153ef410f9ec46ab7e2656413, c7b2dffc3ec34b289e979ca7f90242a8; no voters:
I20260811 22:18:18.968729 936 raft_consensus.cc:2781] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:18.969241 936 raft_consensus.cc:686] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [term 1 LEADER]: Becoming Leader. State: Replica: c7b2dffc3ec34b289e979ca7f90242a8, State: Running, Role: LEADER
I20260811 22:18:18.969849 936 consensus_queue.cc:237] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } }
I20260811 22:18:18.974826 32503 catalog_manager.cc:5495] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7b2dffc3ec34b289e979ca7f90242a8 (127.29.229.3). New cstate: current_term: 1 leader_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } health_report { overall_health: UNKNOWN } } }
I20260811 22:18:19.021600 907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:19.015047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44390 (local address 127.29.229.3:44875)
0811 22:18:19.015318 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:19.015330 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:19.015350 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:18:19.015626 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:19.015630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:19.015720 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:19.015854 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:18:19.016264 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.017101 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.018469 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.018920 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.019022 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:19.019730 (+ 708us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:18:19.020686 (+ 956us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:18:19.020864 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:19.021171 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 22:18:19.021259 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 22:18:19.030441 301 raft_consensus.cc:1261] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413 [term 1 FOLLOWER]: Refusing update from remote peer c7b2dffc3ec34b289e979ca7f90242a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:19.030833 556 raft_consensus.cc:1261] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 1 FOLLOWER]: Refusing update from remote peer c7b2dffc3ec34b289e979ca7f90242a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:19.031500 942 consensus_queue.cc:1035] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:19.031955 936 consensus_queue.cc:1035] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:18:19.041277 397 tablet.cc:2369] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:18:19.042271 945 mvcc.cc:205] Tried to move back new op lower bound from 7317449519217963008 to 7317449518985199616. Current Snapshot: MvccSnapshot[applied={T|T < 7317449519217963008}]
I20260811 22:18:19.047703 947 mvcc.cc:205] Tried to move back new op lower bound from 7317449519217963008 to 7317449518985199616. Current Snapshot: MvccSnapshot[applied={T|T < 7317449519217963008}]
I20260811 22:18:19.063737 32503 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:18:19.064285 32503 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:18:19.071878 32503 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031: Sending DeleteTablet for 3 replicas of tablet c2afc468ea084c4a88dc101298b2f4e6
I20260811 22:18:19.073925 747 tablet_service.cc:1514] Processing DeleteTablet for tablet c2afc468ea084c4a88dc101298b2f4e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:19 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:44388
I20260811 22:18:19.074105 509 tablet_service.cc:1514] Processing DeleteTablet for tablet c2afc468ea084c4a88dc101298b2f4e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:19 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:32978
I20260811 22:18:19.074522 32733 tablet_service.cc:1514] Processing DeleteTablet for tablet c2afc468ea084c4a88dc101298b2f4e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:19 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:37780
I20260811 22:18:19.083501 32502 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:18:19.084952 32502 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:18:19.085330 959 tablet_replica.cc:330] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413: stopping tablet replica
I20260811 22:18:19.086220 959 raft_consensus.cc:2229] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:18:19.098484 508 tablet_service.cc:1467] Processing CreateTablet for tablet 4b8fbbec1a454f0f9246eb2b0e678372 (DEFAULT_TABLE table=test-table [id=66d577d698f94156a6fe06683c5bbdee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:19.099398 508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b8fbbec1a454f0f9246eb2b0e678372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:19.107306 745 tablet_service.cc:1467] Processing CreateTablet for tablet 4b8fbbec1a454f0f9246eb2b0e678372 (DEFAULT_TABLE table=test-table [id=66d577d698f94156a6fe06683c5bbdee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:19.108016 745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b8fbbec1a454f0f9246eb2b0e678372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:19.108153 32733 tablet_service.cc:1467] Processing CreateTablet for tablet 4b8fbbec1a454f0f9246eb2b0e678372 (DEFAULT_TABLE table=test-table [id=66d577d698f94156a6fe06683c5bbdee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:19.108955 32733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b8fbbec1a454f0f9246eb2b0e678372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:19.116513 919 tablet_bootstrap.cc:492] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8: Bootstrap starting.
I20260811 22:18:19.118216 919 tablet_bootstrap.cc:654] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:19.118599 959 raft_consensus.cc:2258] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:19.119731 958 tablet_replica.cc:330] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8: stopping tablet replica
I20260811 22:18:19.120322 958 raft_consensus.cc:2229] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:18:19.121359 958 raft_consensus.cc:2258] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:19.123603 922 tablet_bootstrap.cc:492] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62: Bootstrap starting.
I20260811 22:18:19.126307 957 tablet_replica.cc:330] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62: stopping tablet replica
I20260811 22:18:19.126780 957 raft_consensus.cc:2229] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:18:19.126929 922 tablet_bootstrap.cc:654] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:19.127381 957 raft_consensus.cc:2258] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:19.128908 929 tablet_bootstrap.cc:492] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413: Bootstrap starting.
I20260811 22:18:19.130669 929 tablet_bootstrap.cc:654] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:19.130970 957 ts_tablet_manager.cc:1905] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:19.132135 919 tablet_bootstrap.cc:492] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8: No bootstrap required, opened a new log
I20260811 22:18:19.132325 919 ts_tablet_manager.cc:1397] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.007s
I20260811 22:18:19.133045 958 ts_tablet_manager.cc:1905] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:19.133388 919 raft_consensus.cc:348] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.133711 919 raft_consensus.cc:374] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:19.133809 919 raft_consensus.cc:729] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7b2dffc3ec34b289e979ca7f90242a8, State: Initialized, Role: FOLLOWER
I20260811 22:18:19.134114 919 consensus_queue.cc:260] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.134814 919 ts_tablet_manager.cc:1428] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:18:19.136364 929 tablet_bootstrap.cc:492] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413: No bootstrap required, opened a new log
I20260811 22:18:19.136531 929 ts_tablet_manager.cc:1397] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260811 22:18:19.137518 929 raft_consensus.cc:348] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.137780 929 raft_consensus.cc:374] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:19.137892 929 raft_consensus.cc:729] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 139832b153ef410f9ec46ab7e2656413, State: Initialized, Role: FOLLOWER
I20260811 22:18:19.138125 922 tablet_bootstrap.cc:492] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62: No bootstrap required, opened a new log
I20260811 22:18:19.138528 922 ts_tablet_manager.cc:1397] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.008s
I20260811 22:18:19.138408 929 consensus_queue.cc:260] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.139603 922 raft_consensus.cc:348] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.139791 929 ts_tablet_manager.cc:1428] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:18:19.139863 922 raft_consensus.cc:374] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:19.139981 922 raft_consensus.cc:729] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0080c63ee5974e5d9f2ffe86a1876f62, State: Initialized, Role: FOLLOWER
I20260811 22:18:19.140305 922 consensus_queue.cc:260] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.140928 922 ts_tablet_manager.cc:1428] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 22:18:19.141021 959 ts_tablet_manager.cc:1905] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:19.148348 958 ts_tablet_manager.cc:1918] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:18:19.148597 958 log.cc:1196] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/c2afc468ea084c4a88dc101298b2f4e6
I20260811 22:18:19.149451 958 ts_tablet_manager.cc:1939] T c2afc468ea084c4a88dc101298b2f4e6 P c7b2dffc3ec34b289e979ca7f90242a8: Deleting consensus metadata
I20260811 22:18:19.149825 959 ts_tablet_manager.cc:1918] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:18:19.150049 959 log.cc:1196] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-0/wal/wals/c2afc468ea084c4a88dc101298b2f4e6
I20260811 22:18:19.150611 959 ts_tablet_manager.cc:1939] T c2afc468ea084c4a88dc101298b2f4e6 P 139832b153ef410f9ec46ab7e2656413: Deleting consensus metadata
I20260811 22:18:19.152096 32480 catalog_manager.cc:4841] TS c7b2dffc3ec34b289e979ca7f90242a8 (127.29.229.3:44875): tablet c2afc468ea084c4a88dc101298b2f4e6 (table test-table [id=670522ebd9c040c0aed0cd29c5442db3]) successfully deleted
I20260811 22:18:19.154621 32480 catalog_manager.cc:4841] TS 139832b153ef410f9ec46ab7e2656413 (127.29.229.1:38075): tablet c2afc468ea084c4a88dc101298b2f4e6 (table test-table [id=670522ebd9c040c0aed0cd29c5442db3]) successfully deleted
I20260811 22:18:19.154793 957 ts_tablet_manager.cc:1918] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:18:19.155004 957 log.cc:1196] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/c2afc468ea084c4a88dc101298b2f4e6
I20260811 22:18:19.155642 957 ts_tablet_manager.cc:1939] T c2afc468ea084c4a88dc101298b2f4e6 P 0080c63ee5974e5d9f2ffe86a1876f62: Deleting consensus metadata
I20260811 22:18:19.157744 32480 catalog_manager.cc:4841] TS 0080c63ee5974e5d9f2ffe86a1876f62 (127.29.229.2:44635): tablet c2afc468ea084c4a88dc101298b2f4e6 (table test-table [id=670522ebd9c040c0aed0cd29c5442db3]) successfully deleted
I20260811 22:18:19.229880 939 raft_consensus.cc:482] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:18:19.230134 939 raft_consensus.cc:504] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.232227 939 leader_election.cc:288] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0080c63ee5974e5d9f2ffe86a1876f62 (127.29.229.2:44635), c7b2dffc3ec34b289e979ca7f90242a8 (127.29.229.3:44875)
I20260811 22:18:19.237790 936 raft_consensus.cc:482] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:18:19.237998 936 raft_consensus.cc:504] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.238750 936 leader_election.cc:288] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 139832b153ef410f9ec46ab7e2656413 (127.29.229.1:38075), 0080c63ee5974e5d9f2ffe86a1876f62 (127.29.229.2:44635)
I20260811 22:18:19.241184 909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:19.232677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:43243 (local address 127.29.229.2:44635)
0811 22:18:19.232906 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:19.232914 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:19.233541 (+ 627us) server_negotiation.cc:407] Connection header received
0811 22:18:19.233757 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:19.233773 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:19.233891 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:19.234038 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:19.235023 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.236214 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.238554 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.239765 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.239917 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:19.240013 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:19.240682 (+ 669us) server_negotiation.cc:299] Negotiation successful
0811 22:18:19.240850 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:18:19.241966 964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:19.232683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:44635 (local address 127.29.229.1:43243)
0811 22:18:19.233342 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:19.233364 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:19.233519 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:19.234325 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:19.234331 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:19.234374 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:19.234812 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:19.234827 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.236370 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.236376 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:19.238359 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:19.238374 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.240243 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.240250 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:19.240430 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:19.240497 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 22:18:19.241369 (+ 872us) client_negotiation.cc:240] Negotiation successful
0811 22:18:19.241523 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":53,"spinlock_wait_cycles":13312,"thread_start_us":372,"threads_started":1}
I20260811 22:18:19.243333 907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:19.234302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:47227 (local address 127.29.229.3:44875)
0811 22:18:19.234508 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:19.234516 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:19.234536 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:18:19.234651 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:19.234656 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:19.234735 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:19.234846 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:19.235406 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.236318 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.239191 (+ 2873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.241537 (+ 2346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.241767 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:19.241874 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:19.242766 (+ 892us) server_negotiation.cc:299] Negotiation successful
0811 22:18:19.243030 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 22:18:19.243491 556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8fbbec1a454f0f9246eb2b0e678372" candidate_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" is_pre_election: true
I20260811 22:18:19.243746 966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:19.233541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:44875 (local address 127.29.229.1:47227)
0811 22:18:19.234291 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:19.234309 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:19.234449 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:19.234905 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:19.234909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:19.234934 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:19.235234 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:19.235243 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.236459 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.236467 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:19.238987 (+ 2520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:19.239002 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.242349 (+ 3347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.242365 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:19.242561 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:19.242596 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 22:18:19.242696 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 22:18:19.242839 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":26,"spinlock_wait_cycles":11008,"thread_start_us":419,"threads_started":1}
I20260811 22:18:19.243777 556 raft_consensus.cc:2454] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7b2dffc3ec34b289e979ca7f90242a8 in term 0.
I20260811 22:18:19.244259 668 leader_election.cc:302] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0080c63ee5974e5d9f2ffe86a1876f62, c7b2dffc3ec34b289e979ca7f90242a8; no voters:
I20260811 22:18:19.244269 301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8fbbec1a454f0f9246eb2b0e678372" candidate_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139832b153ef410f9ec46ab7e2656413" is_pre_election: true
I20260811 22:18:19.244561 301 raft_consensus.cc:2454] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7b2dffc3ec34b289e979ca7f90242a8 in term 0.
I20260811 22:18:19.244848 556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8fbbec1a454f0f9246eb2b0e678372" candidate_uuid: "139832b153ef410f9ec46ab7e2656413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" is_pre_election: true
I20260811 22:18:19.245067 556 raft_consensus.cc:2454] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 139832b153ef410f9ec46ab7e2656413 in term 0.
I20260811 22:18:19.245208 784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8fbbec1a454f0f9246eb2b0e678372" candidate_uuid: "139832b153ef410f9ec46ab7e2656413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" is_pre_election: true
I20260811 22:18:19.245416 936 raft_consensus.cc:2781] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:18:19.245558 936 raft_consensus.cc:482] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:18:19.245657 936 raft_consensus.cc:3037] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:19.245617 32655 leader_election.cc:302] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0080c63ee5974e5d9f2ffe86a1876f62, 139832b153ef410f9ec46ab7e2656413; no voters:
I20260811 22:18:19.246002 939 raft_consensus.cc:2781] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:18:19.246124 939 raft_consensus.cc:482] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:18:19.246223 939 raft_consensus.cc:3037] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:19.248847 939 raft_consensus.cc:504] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.248912 936 raft_consensus.cc:504] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.249543 784 raft_consensus.cc:2379] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 139832b153ef410f9ec46ab7e2656413 in current term 1: Already voted for candidate c7b2dffc3ec34b289e979ca7f90242a8 in this term.
I20260811 22:18:19.249594 939 leader_election.cc:288] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413 [CANDIDATE]: Term 1 election: Requested vote from peers 0080c63ee5974e5d9f2ffe86a1876f62 (127.29.229.2:44635), c7b2dffc3ec34b289e979ca7f90242a8 (127.29.229.3:44875)
I20260811 22:18:19.250146 301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8fbbec1a454f0f9246eb2b0e678372" candidate_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "139832b153ef410f9ec46ab7e2656413"
I20260811 22:18:19.250242 936 leader_election.cc:288] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [CANDIDATE]: Term 1 election: Requested vote from peers 139832b153ef410f9ec46ab7e2656413 (127.29.229.1:38075), 0080c63ee5974e5d9f2ffe86a1876f62 (127.29.229.2:44635)
I20260811 22:18:19.250425 301 raft_consensus.cc:2379] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c7b2dffc3ec34b289e979ca7f90242a8 in current term 1: Already voted for candidate 139832b153ef410f9ec46ab7e2656413 in this term.
I20260811 22:18:19.250862 556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8fbbec1a454f0f9246eb2b0e678372" candidate_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0080c63ee5974e5d9f2ffe86a1876f62"
I20260811 22:18:19.251073 556 raft_consensus.cc:3037] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:19.251292 555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8fbbec1a454f0f9246eb2b0e678372" candidate_uuid: "139832b153ef410f9ec46ab7e2656413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0080c63ee5974e5d9f2ffe86a1876f62"
I20260811 22:18:19.251945 784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8fbbec1a454f0f9246eb2b0e678372" candidate_uuid: "139832b153ef410f9ec46ab7e2656413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7b2dffc3ec34b289e979ca7f90242a8"
I20260811 22:18:19.252209 784 raft_consensus.cc:2379] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 139832b153ef410f9ec46ab7e2656413 in current term 1: Already voted for candidate c7b2dffc3ec34b289e979ca7f90242a8 in this term.
I20260811 22:18:19.252641 32655 leader_election.cc:302] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 139832b153ef410f9ec46ab7e2656413; no voters: 0080c63ee5974e5d9f2ffe86a1876f62, c7b2dffc3ec34b289e979ca7f90242a8
I20260811 22:18:19.253108 939 raft_consensus.cc:2726] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260811 22:18:19.253975 556 raft_consensus.cc:2454] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7b2dffc3ec34b289e979ca7f90242a8 in term 1.
I20260811 22:18:19.254343 668 leader_election.cc:302] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0080c63ee5974e5d9f2ffe86a1876f62, c7b2dffc3ec34b289e979ca7f90242a8; no voters: 139832b153ef410f9ec46ab7e2656413
I20260811 22:18:19.254707 936 raft_consensus.cc:2781] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:19.254904 936 raft_consensus.cc:686] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [term 1 LEADER]: Becoming Leader. State: Replica: c7b2dffc3ec34b289e979ca7f90242a8, State: Running, Role: LEADER
I20260811 22:18:19.255349 936 consensus_queue.cc:237] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.259238 32502 catalog_manager.cc:5495] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7b2dffc3ec34b289e979ca7f90242a8 (127.29.229.3). New cstate: current_term: 1 leader_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } health_report { overall_health: HEALTHY } } }
Aug 11 22:18:19 dist-test-slave-jq4t krb5kdc[32411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486696, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:19.324735 978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:19.307769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:44989 (local address 127.0.0.1:33130)
0811 22:18:19.308728 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:19.308750 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:19.308931 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:19.309485 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:19.309490 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:19.310449 (+ 959us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:19.310786 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:19.310812 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.312088 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.312095 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:19.313260 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:19.313273 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.314044 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.314050 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:19.314220 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:19.315040 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:19.315059 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:19.317586 (+ 2527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:19.321714 (+ 4128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:19.321726 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:19.321732 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:19.322478 (+ 746us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:19.322957 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:19.322961 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:19.322964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:19.323113 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:19.323789 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:19.323796 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:19.323984 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 22:18:19.324128 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 22:18:19.324360 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"thread_start_us":613,"threads_started":1}
I20260811 22:18:19.325168 977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:19.307682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33130 (local address 127.29.229.62:44989)
0811 22:18:19.308627 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:19.308634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:19.308919 (+ 285us) server_negotiation.cc:407] Connection header received
0811 22:18:19.309094 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:19.309098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:19.309268 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:19.309391 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:19.310991 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.311938 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.313412 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.313833 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.313980 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:19.317814 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:19.317831 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:19.317836 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:19.317873 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:19.321558 (+ 3685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:19.322622 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:19.322629 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:19.322633 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:19.322812 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:19.323238 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:19.323246 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:19.323250 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:19.323629 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:19.323784 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:19.324279 (+ 495us) server_negotiation.cc:299] Negotiation successful
0811 22:18:19.324579 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":458,"threads_started":1}
I20260811 22:18:19.331889 32502 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:33130:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:18:19.333295 32502 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:18:19.340847 32733 tablet_service.cc:1467] Processing CreateTablet for tablet 52f2876eecc54ffe988e8b1f79f20052 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=10031cbd225344e0843a292d31129759]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:18:19.341224 509 tablet_service.cc:1467] Processing CreateTablet for tablet 52f2876eecc54ffe988e8b1f79f20052 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=10031cbd225344e0843a292d31129759]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:18:19.341557 32733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52f2876eecc54ffe988e8b1f79f20052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:19.341799 509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52f2876eecc54ffe988e8b1f79f20052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:19.342072 745 tablet_service.cc:1467] Processing CreateTablet for tablet 52f2876eecc54ffe988e8b1f79f20052 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=10031cbd225344e0843a292d31129759]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:18:19.342595 745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52f2876eecc54ffe988e8b1f79f20052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:19.350495 919 tablet_bootstrap.cc:492] T 52f2876eecc54ffe988e8b1f79f20052 P c7b2dffc3ec34b289e979ca7f90242a8: Bootstrap starting.
I20260811 22:18:19.350728 922 tablet_bootstrap.cc:492] T 52f2876eecc54ffe988e8b1f79f20052 P 0080c63ee5974e5d9f2ffe86a1876f62: Bootstrap starting.
I20260811 22:18:19.351886 919 tablet_bootstrap.cc:654] T 52f2876eecc54ffe988e8b1f79f20052 P c7b2dffc3ec34b289e979ca7f90242a8: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:19.352520 922 tablet_bootstrap.cc:654] T 52f2876eecc54ffe988e8b1f79f20052 P 0080c63ee5974e5d9f2ffe86a1876f62: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:19.353492 929 tablet_bootstrap.cc:492] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413: Bootstrap starting.
I20260811 22:18:19.355857 929 tablet_bootstrap.cc:654] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413: Neither blocks nor log segments found. Creating new log.
W20260811 22:18:19.363894 636 tablet.cc:2369] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:18:19.364207 919 tablet_bootstrap.cc:492] T 52f2876eecc54ffe988e8b1f79f20052 P c7b2dffc3ec34b289e979ca7f90242a8: No bootstrap required, opened a new log
I20260811 22:18:19.364506 919 ts_tablet_manager.cc:1397] T 52f2876eecc54ffe988e8b1f79f20052 P c7b2dffc3ec34b289e979ca7f90242a8: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.001s
I20260811 22:18:19.365818 919 raft_consensus.cc:348] T 52f2876eecc54ffe988e8b1f79f20052 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.366483 919 raft_consensus.cc:374] T 52f2876eecc54ffe988e8b1f79f20052 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:19.366678 919 raft_consensus.cc:729] T 52f2876eecc54ffe988e8b1f79f20052 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7b2dffc3ec34b289e979ca7f90242a8, State: Initialized, Role: FOLLOWER
I20260811 22:18:19.367113 919 consensus_queue.cc:260] T 52f2876eecc54ffe988e8b1f79f20052 P c7b2dffc3ec34b289e979ca7f90242a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.367864 936 tablet_replica.cc:439] T 52f2876eecc54ffe988e8b1f79f20052 P c7b2dffc3ec34b289e979ca7f90242a8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } } }
I20260811 22:18:19.368256 936 tablet_replica.cc:442] T 52f2876eecc54ffe988e8b1f79f20052 P c7b2dffc3ec34b289e979ca7f90242a8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:18:19.368453 929 tablet_bootstrap.cc:492] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413: No bootstrap required, opened a new log
I20260811 22:18:19.368665 929 ts_tablet_manager.cc:1397] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.007s
I20260811 22:18:19.368664 922 tablet_bootstrap.cc:492] T 52f2876eecc54ffe988e8b1f79f20052 P 0080c63ee5974e5d9f2ffe86a1876f62: No bootstrap required, opened a new log
I20260811 22:18:19.368884 922 ts_tablet_manager.cc:1397] T 52f2876eecc54ffe988e8b1f79f20052 P 0080c63ee5974e5d9f2ffe86a1876f62: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.006s
I20260811 22:18:19.369495 929 raft_consensus.cc:348] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.369760 929 raft_consensus.cc:374] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:19.369858 929 raft_consensus.cc:729] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 139832b153ef410f9ec46ab7e2656413, State: Initialized, Role: FOLLOWER
I20260811 22:18:19.370373 929 consensus_queue.cc:260] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.370944 919 ts_tablet_manager.cc:1428] T 52f2876eecc54ffe988e8b1f79f20052 P c7b2dffc3ec34b289e979ca7f90242a8: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260811 22:18:19.369726 922 raft_consensus.cc:348] T 52f2876eecc54ffe988e8b1f79f20052 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.371096 939 tablet_replica.cc:439] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } } }
I20260811 22:18:19.371390 922 raft_consensus.cc:374] T 52f2876eecc54ffe988e8b1f79f20052 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:19.371417 939 tablet_replica.cc:442] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:18:19.371486 922 raft_consensus.cc:729] T 52f2876eecc54ffe988e8b1f79f20052 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0080c63ee5974e5d9f2ffe86a1876f62, State: Initialized, Role: FOLLOWER
I20260811 22:18:19.371726 922 consensus_queue.cc:260] T 52f2876eecc54ffe988e8b1f79f20052 P 0080c63ee5974e5d9f2ffe86a1876f62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.372383 922 ts_tablet_manager.cc:1428] T 52f2876eecc54ffe988e8b1f79f20052 P 0080c63ee5974e5d9f2ffe86a1876f62: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260811 22:18:19.372270 938 tablet_replica.cc:439] T 52f2876eecc54ffe988e8b1f79f20052 P 0080c63ee5974e5d9f2ffe86a1876f62: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } } }
I20260811 22:18:19.372740 938 tablet_replica.cc:442] T 52f2876eecc54ffe988e8b1f79f20052 P 0080c63ee5974e5d9f2ffe86a1876f62: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:18:19.374498 929 ts_tablet_manager.cc:1428] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260811 22:18:19.730391 939 raft_consensus.cc:482] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:18:19.730787 939 raft_consensus.cc:504] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.731688 939 leader_election.cc:288] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0080c63ee5974e5d9f2ffe86a1876f62 (127.29.229.2:44635), c7b2dffc3ec34b289e979ca7f90242a8 (127.29.229.3:44875)
I20260811 22:18:19.732379 556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52f2876eecc54ffe988e8b1f79f20052" candidate_uuid: "139832b153ef410f9ec46ab7e2656413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" is_pre_election: true
I20260811 22:18:19.732527 784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52f2876eecc54ffe988e8b1f79f20052" candidate_uuid: "139832b153ef410f9ec46ab7e2656413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" is_pre_election: true
I20260811 22:18:19.732702 556 raft_consensus.cc:2454] T 52f2876eecc54ffe988e8b1f79f20052 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 139832b153ef410f9ec46ab7e2656413 in term 0.
I20260811 22:18:19.733067 784 raft_consensus.cc:2454] T 52f2876eecc54ffe988e8b1f79f20052 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 139832b153ef410f9ec46ab7e2656413 in term 0.
I20260811 22:18:19.733215 32655 leader_election.cc:302] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0080c63ee5974e5d9f2ffe86a1876f62, 139832b153ef410f9ec46ab7e2656413; no voters:
I20260811 22:18:19.733584 939 raft_consensus.cc:2781] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:18:19.733701 939 raft_consensus.cc:482] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:18:19.733822 939 raft_consensus.cc:3037] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:19.736205 939 raft_consensus.cc:504] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.736915 939 leader_election.cc:288] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 [CANDIDATE]: Term 1 election: Requested vote from peers 0080c63ee5974e5d9f2ffe86a1876f62 (127.29.229.2:44635), c7b2dffc3ec34b289e979ca7f90242a8 (127.29.229.3:44875)
I20260811 22:18:19.737776 784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52f2876eecc54ffe988e8b1f79f20052" candidate_uuid: "139832b153ef410f9ec46ab7e2656413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7b2dffc3ec34b289e979ca7f90242a8"
I20260811 22:18:19.737852 556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52f2876eecc54ffe988e8b1f79f20052" candidate_uuid: "139832b153ef410f9ec46ab7e2656413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0080c63ee5974e5d9f2ffe86a1876f62"
I20260811 22:18:19.738024 784 raft_consensus.cc:3037] T 52f2876eecc54ffe988e8b1f79f20052 P c7b2dffc3ec34b289e979ca7f90242a8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:19.738118 556 raft_consensus.cc:3037] T 52f2876eecc54ffe988e8b1f79f20052 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:19.740231 556 raft_consensus.cc:2454] T 52f2876eecc54ffe988e8b1f79f20052 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 139832b153ef410f9ec46ab7e2656413 in term 1.
I20260811 22:18:19.740247 784 raft_consensus.cc:2454] T 52f2876eecc54ffe988e8b1f79f20052 P c7b2dffc3ec34b289e979ca7f90242a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 139832b153ef410f9ec46ab7e2656413 in term 1.
I20260811 22:18:19.740784 32655 leader_election.cc:302] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 139832b153ef410f9ec46ab7e2656413, c7b2dffc3ec34b289e979ca7f90242a8; no voters:
I20260811 22:18:19.741132 939 raft_consensus.cc:2781] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:19.741644 939 raft_consensus.cc:686] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 [term 1 LEADER]: Becoming Leader. State: Replica: 139832b153ef410f9ec46ab7e2656413, State: Running, Role: LEADER
I20260811 22:18:19.742074 939 consensus_queue.cc:237] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } }
I20260811 22:18:19.744333 983 tablet_replica.cc:439] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413: TxnStatusTablet state changed. Reason: New leader 139832b153ef410f9ec46ab7e2656413. Latest consensus state: current_term: 1 leader_uuid: "139832b153ef410f9ec46ab7e2656413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } } }
I20260811 22:18:19.744657 983 tablet_replica.cc:442] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:18:19.745247 985 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:18:19.745574 985 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:18:19.746338 32503 catalog_manager.cc:5495] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 reported cstate change: term changed from 0 to 1, leader changed from <none> to 139832b153ef410f9ec46ab7e2656413 (127.29.229.1). New cstate: current_term: 1 leader_uuid: "139832b153ef410f9ec46ab7e2656413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } health_report { overall_health: UNKNOWN } } }
I20260811 22:18:19.826958 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:19.819465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37786 (local address 127.29.229.1:38075)
0811 22:18:19.820067 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:19.820071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:19.820089 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:18:19.820170 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:19.820174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:19.820265 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:19.820400 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:18:19.820989 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.821899 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.824000 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.824574 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.824694 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:19.825403 (+ 709us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:18:19.826127 (+ 724us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:18:19.826253 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:19.826553 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 22:18:19.826640 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":326,"threads_started":1}
I20260811 22:18:19.826994 978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:19.819336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:38075 (local address 127.0.0.1:37786)
0811 22:18:19.819603 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:19.819627 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:19.819775 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:19.820453 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:19.820458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:19.820504 (+ 46us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:18:19.820744 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:19.820755 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.822038 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.822045 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:19.823728 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:19.823744 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.824855 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.824862 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:19.825065 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:19.825149 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:18:19.826251 (+ 1102us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:18:19.826358 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 22:18:19.826477 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 22:18:19.826649 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 22:18:19.832626 556 raft_consensus.cc:1261] T 52f2876eecc54ffe988e8b1f79f20052 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 1 FOLLOWER]: Refusing update from remote peer 139832b153ef410f9ec46ab7e2656413: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:19.832924 784 raft_consensus.cc:1261] T 52f2876eecc54ffe988e8b1f79f20052 P c7b2dffc3ec34b289e979ca7f90242a8 [term 1 FOLLOWER]: Refusing update from remote peer 139832b153ef410f9ec46ab7e2656413: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:19.833487 983 consensus_queue.cc:1035] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:19.834043 939 consensus_queue.cc:1035] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:19.838819 938 tablet_replica.cc:439] T 52f2876eecc54ffe988e8b1f79f20052 P 0080c63ee5974e5d9f2ffe86a1876f62: TxnStatusTablet state changed. Reason: New leader 139832b153ef410f9ec46ab7e2656413. Latest consensus state: current_term: 1 leader_uuid: "139832b153ef410f9ec46ab7e2656413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } } }
I20260811 22:18:19.839340 938 tablet_replica.cc:442] T 52f2876eecc54ffe988e8b1f79f20052 P 0080c63ee5974e5d9f2ffe86a1876f62: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:18:19.843235 992 mvcc.cc:205] Tried to move back new op lower bound from 7317449522506088448 to 7317449522147528704. Current Snapshot: MvccSnapshot[applied={T|T < 7317449522506088448}]
I20260811 22:18:19.844352 936 tablet_replica.cc:439] T 52f2876eecc54ffe988e8b1f79f20052 P c7b2dffc3ec34b289e979ca7f90242a8: TxnStatusTablet state changed. Reason: New leader 139832b153ef410f9ec46ab7e2656413. Latest consensus state: current_term: 1 leader_uuid: "139832b153ef410f9ec46ab7e2656413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } } }
I20260811 22:18:19.844700 936 tablet_replica.cc:442] T 52f2876eecc54ffe988e8b1f79f20052 P c7b2dffc3ec34b289e979ca7f90242a8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:18:19.853066 983 tablet_replica.cc:439] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "139832b153ef410f9ec46ab7e2656413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } } }
I20260811 22:18:19.853421 983 tablet_replica.cc:442] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:18:19.854734 983 tablet_replica.cc:439] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "139832b153ef410f9ec46ab7e2656413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } } }
I20260811 22:18:19.855350 983 tablet_replica.cc:442] T 52f2876eecc54ffe988e8b1f79f20052 P 139832b153ef410f9ec46ab7e2656413: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:18:19.857565 936 tablet_replica.cc:439] T 52f2876eecc54ffe988e8b1f79f20052 P c7b2dffc3ec34b289e979ca7f90242a8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "139832b153ef410f9ec46ab7e2656413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } } }
I20260811 22:18:19.857897 936 tablet_replica.cc:442] T 52f2876eecc54ffe988e8b1f79f20052 P c7b2dffc3ec34b289e979ca7f90242a8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:18:19.871992 938 tablet_replica.cc:439] T 52f2876eecc54ffe988e8b1f79f20052 P 0080c63ee5974e5d9f2ffe86a1876f62: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "139832b153ef410f9ec46ab7e2656413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 } } peers { permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 } } peers { permanent_uuid: "c7b2dffc3ec34b289e979ca7f90242a8" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44875 } } }
I20260811 22:18:19.872562 938 tablet_replica.cc:442] T 52f2876eecc54ffe988e8b1f79f20052 P 0080c63ee5974e5d9f2ffe86a1876f62: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:18:19.879242 936 consensus_queue.cc:1035] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "139832b153ef410f9ec46ab7e2656413" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:19.895406 936 consensus_queue.cc:1035] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0080c63ee5974e5d9f2ffe86a1876f62" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 44635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:19.902676 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:19.892431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:38087 (local address 127.29.229.1:38075)
0811 22:18:19.892653 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:19.892662 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:19.892757 (+ 95us) server_negotiation.cc:407] Connection header received
0811 22:18:19.893149 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:19.893154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:19.893255 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:19.893322 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:18:19.894035 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.895342 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.898538 (+ 3196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.899323 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.899447 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:19.901631 (+ 2184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:18:19.901879 (+ 248us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:18:19.901957 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:19.902198 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 22:18:19.902323 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 22:18:19.905442 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:19.891845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:38075 (local address 127.29.229.3:38087)
0811 22:18:19.892646 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:19.892668 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:19.892905 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:19.893544 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:19.893548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:19.893561 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:18:19.893845 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:19.893858 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.895538 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.895543 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:19.898159 (+ 2616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:19.898197 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.900669 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.900680 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:19.901037 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:19.901254 (+ 217us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:18:19.902053 (+ 799us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:18:19.902063 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 22:18:19.904343 (+ 2280us) client_negotiation.cc:240] Negotiation successful
0811 22:18:19.904596 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":669,"threads_started":1}
I20260811 22:18:19.964725 1020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:19.942511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:33353 (local address 127.29.229.3:44875)
0811 22:18:19.957144 (+ 14633us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:19.957152 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:19.957182 (+ 30us) server_negotiation.cc:407] Connection header received
0811 22:18:19.957259 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:19.957264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:19.957360 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:19.957487 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:18:19.958232 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.959314 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.962270 (+ 2956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.962986 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.963120 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:19.963759 (+ 639us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:18:19.963936 (+ 177us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:18:19.964039 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:19.964162 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 22:18:19.964244 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14460,"thread_start_us":14300,"threads_started":1}
I20260811 22:18:19.964990 1023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:19.942556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:44875 (local address 127.29.229.1:33353)
0811 22:18:19.953433 (+ 10877us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:19.953464 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:19.953718 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:19.957524 (+ 3806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:19.957533 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:19.957555 (+ 22us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:18:19.958045 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:19.958058 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.959448 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.959455 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:19.962105 (+ 2650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:19.962124 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:19.963181 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:19.963188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:19.963386 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:19.963488 (+ 102us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:18:19.964043 (+ 555us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:18:19.964057 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 22:18:19.964147 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 22:18:19.964313 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10508,"thread_start_us":10155,"threads_started":1}
I20260811 22:18:20.006603 32500 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:18:20.006917 32500 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:18:20.010924 32500 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 95e81b8a7ec9467bb3d3a9bc0c0f0031: Sending DeleteTablet for 3 replicas of tablet 4b8fbbec1a454f0f9246eb2b0e678372
I20260811 22:18:20.012015 32733 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b8fbbec1a454f0f9246eb2b0e678372 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:20 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:37780
I20260811 22:18:20.012205 509 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b8fbbec1a454f0f9246eb2b0e678372 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:20 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:32978
I20260811 22:18:20.014153 745 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b8fbbec1a454f0f9246eb2b0e678372 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:20 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:44388
I20260811 22:18:20.020267 1027 tablet_replica.cc:330] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413: stopping tablet replica
I20260811 22:18:20.020728 1027 raft_consensus.cc:2229] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:18:20.021472 1027 raft_consensus.cc:2258] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:20.023360 1027 ts_tablet_manager.cc:1905] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:20.033335 1034 tablet_replica.cc:330] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8: stopping tablet replica
I20260811 22:18:20.033695 1034 raft_consensus.cc:2229] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:18:20.034328 1034 raft_consensus.cc:2258] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:20.035966 1034 ts_tablet_manager.cc:1905] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:20.037901 1027 ts_tablet_manager.cc:1918] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:18:20.038153 1027 log.cc:1196] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-0/wal/wals/4b8fbbec1a454f0f9246eb2b0e678372
I20260811 22:18:20.038817 1027 ts_tablet_manager.cc:1939] T 4b8fbbec1a454f0f9246eb2b0e678372 P 139832b153ef410f9ec46ab7e2656413: Deleting consensus metadata
I20260811 22:18:20.040294 32480 catalog_manager.cc:4841] TS 139832b153ef410f9ec46ab7e2656413 (127.29.229.1:38075): tablet 4b8fbbec1a454f0f9246eb2b0e678372 (table test-table [id=66d577d698f94156a6fe06683c5bbdee]) successfully deleted
I20260811 22:18:20.044621 1028 tablet_replica.cc:330] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62: stopping tablet replica
I20260811 22:18:20.044883 1028 raft_consensus.cc:2229] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 1 FOLLOWER]: Raft consensus shutting down.
Aug 11 22:18:20 dist-test-slave-jq4t krb5kdc[32411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486698, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:18:20.045217 1028 raft_consensus.cc:2258] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:20.046618 1028 ts_tablet_manager.cc:1905] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:20.051913 1034 ts_tablet_manager.cc:1918] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:18:20.052268 1034 log.cc:1196] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/4b8fbbec1a454f0f9246eb2b0e678372
I20260811 22:18:20.053169 1034 ts_tablet_manager.cc:1939] T 4b8fbbec1a454f0f9246eb2b0e678372 P c7b2dffc3ec34b289e979ca7f90242a8: Deleting consensus metadata
I20260811 22:18:20.054574 32480 catalog_manager.cc:4841] TS c7b2dffc3ec34b289e979ca7f90242a8 (127.29.229.3:44875): tablet 4b8fbbec1a454f0f9246eb2b0e678372 (table test-table [id=66d577d698f94156a6fe06683c5bbdee]) successfully deleted
I20260811 22:18:20.055490 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:20.023582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37798 (local address 127.29.229.1:38075)
0811 22:18:20.023830 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:20.023839 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:20.031783 (+ 7944us) server_negotiation.cc:407] Connection header received
0811 22:18:20.032466 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:20.032501 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:20.032610 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:20.034729 (+ 2119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:20.035223 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:20.036653 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:20.038984 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:20.040182 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:20.040332 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:20.046415 (+ 6083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:20.046432 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:20.046451 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:20.046502 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:20.050994 (+ 4492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:20.052018 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:20.052027 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:20.052030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:20.052253 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:20.052765 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:20.052769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:20.052773 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:20.053561 (+ 788us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:20.054030 (+ 469us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:20.054758 (+ 728us) server_negotiation.cc:299] Negotiation successful
0811 22:18:20.055057 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 22:18:20.055919 1028 ts_tablet_manager.cc:1918] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:18:20.056061 1028 log.cc:1196] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/4b8fbbec1a454f0f9246eb2b0e678372
I20260811 22:18:20.056531 1028 ts_tablet_manager.cc:1939] T 4b8fbbec1a454f0f9246eb2b0e678372 P 0080c63ee5974e5d9f2ffe86a1876f62: Deleting consensus metadata
W20260811 22:18:20.057243 388 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37798
I20260811 22:18:20.058054 32480 catalog_manager.cc:4841] TS 0080c63ee5974e5d9f2ffe86a1876f62 (127.29.229.2:44635): tablet 4b8fbbec1a454f0f9246eb2b0e678372 (table test-table [id=66d577d698f94156a6fe06683c5bbdee]) successfully deleted
I20260811 22:18:20.086284 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:20.060998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33134 (local address 127.29.229.62:44989)
0811 22:18:20.069335 (+ 8337us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:20.069341 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:20.069380 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:18:20.069476 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:20.069481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:20.069606 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:20.069847 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:20.071931 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:20.072972 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:20.074612 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:20.075123 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:20.075283 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:20.078293 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:20.078298 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:20.078302 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:20.078341 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:20.082386 (+ 4045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:20.083471 (+ 1085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:20.083477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:20.083482 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:20.083650 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:20.084379 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:20.084385 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:20.084388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:20.084851 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:20.085134 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:20.085601 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 22:18:20.085874 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8138,"thread_start_us":7856,"threads_started":1}
W20260811 22:18:20.087055 32500 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33134
I20260811 22:18:20.089263 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 32620
I20260811 22:18:20.106035 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:20.090966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37814 (local address 127.29.229.1:38075)
0811 22:18:20.091261 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:20.091270 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:20.091545 (+ 275us) server_negotiation.cc:407] Connection header received
0811 22:18:20.091745 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:20.091751 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:20.091827 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:20.091939 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:20.093221 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:20.094019 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:20.095247 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:20.095734 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:20.095875 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:20.099086 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:20.099098 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:20.099102 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:20.099140 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:20.102576 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:20.103462 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:20.103467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:20.103469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:20.103612 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:20.104070 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:20.104075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:20.104077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:20.104404 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:20.104747 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:20.105322 (+ 575us) server_negotiation.cc:299] Negotiation successful
0811 22:18:20.105545 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 22:18:20.106716 388 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37814
W20260811 22:18:20.107333 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 32620: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:18:20.107606 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 32620
I20260811 22:18:20.148645 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 402
Aug 11 22:18:20 dist-test-slave-jq4t krb5kdc[32411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486698, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.229.2@KRBTEST.COM
I20260811 22:18:20.165274 1042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:20.149831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32980 (local address 127.29.229.2:44635)
0811 22:18:20.150399 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:20.150404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:20.150420 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:18:20.150475 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:20.150479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:20.150544 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:20.150668 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:20.151892 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:20.152726 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:20.153960 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:20.154418 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:20.154569 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:20.159079 (+ 4510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:20.159095 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:20.159111 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:20.159159 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:20.162384 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:20.163107 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:20.163115 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:20.163117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:20.163248 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:20.163631 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:20.163636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:20.163640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:20.164038 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:20.164338 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:20.164672 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 22:18:20.164914 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":278,"threads_started":1}
W20260811 22:18:20.166033 628 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32980
W20260811 22:18:20.166726 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 402: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:18:20.166947 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 402
I20260811 22:18:20.197695 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 642
Aug 11 22:18:20 dist-test-slave-jq4t krb5kdc[32411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486698, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.229.3@KRBTEST.COM
I20260811 22:18:20.216710 1020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:20.198949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44406 (local address 127.29.229.3:44875)
0811 22:18:20.199367 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:20.199374 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:20.199402 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:18:20.199468 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:20.199472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:20.199554 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:20.199651 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:20.201205 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:20.202083 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:20.203598 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:20.204183 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:20.204333 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:20.210306 (+ 5973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:20.210326 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:20.210343 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:20.210393 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:20.213718 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:20.214532 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:20.214540 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:20.214542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:20.214692 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:20.215133 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:20.215138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:20.215140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:20.215528 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:20.215815 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:20.216118 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 22:18:20.216304 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
W20260811 22:18:20.217399 874 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44406
W20260811 22:18:20.218058 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 642: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:18:20.218225 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 642
I20260811 22:18:20.255839 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 32441
I20260811 22:18:20.257105 32572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:20.380792 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 32441
2026-08-11T22:18:20Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5649 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:18:20 dist-test-slave-jq4t krb5kdc[1052](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:18:20 dist-test-slave-jq4t krb5kdc[1052](info): set up 2 sockets
Aug 11 22:18:20 dist-test-slave-jq4t krb5kdc[1052](info): commencing operation
krb5kdc: starting...
W20260811 22:18:21.763564 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.326s user 0.001s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:18:21 dist-test-slave-jq4t krb5kdc[1052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:18:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:18:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:18:22.008615 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:43861
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:38689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:43861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:22.361804 1070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:22.362103 1070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:22.362201 1070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:22.369753 1070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:22.369876 1070 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:18:22.369918 1070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:22.369974 1070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:22.370025 1070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:22.370082 1070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:18:22.380391 1070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:38689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:43861
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:43861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:22.381769 1070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:22.383100 1070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:22.395745 1080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:22.395766 1081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:22.395983 1083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:22.396777 1070 server_base.cc:1034] running on GCE node
I20260811 22:18:22.397668 1070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:22.399191 1070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:22.400574 1070 hybrid_clock.cc:648] HybridClock initialized: now 1786486702400446 us; error 39 us; skew 500 ppm
I20260811 22:18:22.400947 1070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:22 dist-test-slave-jq4t krb5kdc[1052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486702, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:22.408074 1070 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:18:22.409843 1070 webserver.cc:460] Webserver started at http://127.29.229.62:41041/ using document root <none> and password file <none>
I20260811 22:18:22.410580 1070 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:22.410672 1070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:22.410982 1070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:22.414213 1070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "c1ea83c38344487f8c0a8f0eff160c51"
format_stamp: "Formatted at 2026-08-11 22:18:22 on dist-test-slave-jq4t"
server_key: "1005f285c01a675904252afcae722e47"
server_key_iv: "ed5e00e65bf975290e84a1074150f52f"
server_key_version: "encryptionkey@0"
I20260811 22:18:22.414979 1070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "c1ea83c38344487f8c0a8f0eff160c51"
format_stamp: "Formatted at 2026-08-11 22:18:22 on dist-test-slave-jq4t"
server_key: "1005f285c01a675904252afcae722e47"
server_key_iv: "ed5e00e65bf975290e84a1074150f52f"
server_key_version: "encryptionkey@0"
I20260811 22:18:22.421272 1070 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 22:18:22.425740 1093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:22.427529 1070 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:18:22.427755 1070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "c1ea83c38344487f8c0a8f0eff160c51"
format_stamp: "Formatted at 2026-08-11 22:18:22 on dist-test-slave-jq4t"
server_key: "1005f285c01a675904252afcae722e47"
server_key_iv: "ed5e00e65bf975290e84a1074150f52f"
server_key_version: "encryptionkey@0"
I20260811 22:18:22.428002 1070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:22.444015 1070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:22.449111 1070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:22.449407 1070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:22.473197 1070 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:43861
I20260811 22:18:22.473235 1191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:43861 every 8 connection(s)
I20260811 22:18:22.474658 1070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:22.478425 1193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:22.484066 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 1070
I20260811 22:18:22.484297 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:18:22.485018 30612 external_mini_cluster.cc:1428] Setting key 3a2fd8afea304d732e0f00d68458046d
I20260811 22:18:22.490500 1193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51: Bootstrap starting.
I20260811 22:18:22.494544 1193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:22.495750 1193 log.cc:824] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:22.498935 1193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51: No bootstrap required, opened a new log
Aug 11 22:18:22 dist-test-slave-jq4t krb5kdc[1052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:22.509902 1193 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ea83c38344487f8c0a8f0eff160c51" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43861 } }
I20260811 22:18:22.510409 1193 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:22.510538 1193 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1ea83c38344487f8c0a8f0eff160c51, State: Initialized, Role: FOLLOWER
I20260811 22:18:22.511318 1193 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ea83c38344487f8c0a8f0eff160c51" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43861 } }
I20260811 22:18:22.511587 1200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:22.487152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47442 (local address 127.29.229.62:43861)
0811 22:18:22.488082 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:22.488098 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:22.488148 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:18:22.490562 (+ 2414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:22.490592 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:22.491121 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:22.491542 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:22.493022 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:22.495107 (+ 2085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:22.496668 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:22.497553 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:22.497741 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:22.501877 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:22.501893 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:22.501913 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:22.501967 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:22.506256 (+ 4289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:22.507009 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:22.507018 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:22.507027 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:22.507265 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:22.507644 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:22.507652 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:22.507657 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:22.508457 (+ 800us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:22.508790 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:22.509325 (+ 535us) server_negotiation.cc:299] Negotiation successful
0811 22:18:22.510079 (+ 754us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":377,"threads_started":1}
I20260811 22:18:22.511665 1193 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:18:22.511796 1193 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:18:22.511996 1193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:22.514665 1193 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ea83c38344487f8c0a8f0eff160c51" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43861 } }
I20260811 22:18:22.515542 1193 leader_election.cc:302] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1ea83c38344487f8c0a8f0eff160c51; no voters:
I20260811 22:18:22.516163 1193 leader_election.cc:288] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:18:22.516330 1206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:22.517095 1206 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51 [term 1 LEADER]: Becoming Leader. State: Replica: c1ea83c38344487f8c0a8f0eff160c51, State: Running, Role: LEADER
I20260811 22:18:22.517675 1206 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ea83c38344487f8c0a8f0eff160c51" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43861 } }
I20260811 22:18:22.518349 1193 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:22.521001 1207 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1ea83c38344487f8c0a8f0eff160c51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ea83c38344487f8c0a8f0eff160c51" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43861 } } }
I20260811 22:18:22.520699 1208 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1ea83c38344487f8c0a8f0eff160c51. Latest consensus state: current_term: 1 leader_uuid: "c1ea83c38344487f8c0a8f0eff160c51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ea83c38344487f8c0a8f0eff160c51" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43861 } } }
I20260811 22:18:22.521757 1207 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:22.521798 1208 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:22.524855 1213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:22.530658 1213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:22.540612 1213 catalog_manager.cc:1347] Generated new cluster ID: dfd4593edd0d48aa840ad19d2f54de9c
I20260811 22:18:22.540755 1213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:22.551672 1213 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:18:22.553094 1213 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:22.566517 1213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51: Generated new TSK 0
I20260811 22:18:22.567533 1213 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
I20260811 22:18:22.628217 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43861
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:38689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:22.993127 1243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:22.993402 1243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:22.993511 1243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:23.000766 1243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:23.000900 1243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:23.000989 1243 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:18:23.001036 1243 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:18:23.001099 1243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:18:23.011166 1243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:38689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.229.62:43861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:23.012646 1243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:23.014001 1243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:23.028854 1255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:23.028925 1252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:23.028824 1251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:23.035614 1243 server_base.cc:1034] running on GCE node
I20260811 22:18:23.036470 1243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:23.037611 1243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:23.038877 1243 hybrid_clock.cc:648] HybridClock initialized: now 1786486703038766 us; error 65 us; skew 500 ppm
I20260811 22:18:23.039247 1243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:23 dist-test-slave-jq4t krb5kdc[1052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486703, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:23.043198 1243 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:18:23.045068 1243 webserver.cc:460] Webserver started at http://127.29.229.1:36677/ using document root <none> and password file <none>
I20260811 22:18:23.045887 1243 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:23.046028 1243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:23.046386 1243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:23.049551 1243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "9a977e616b4c41bf8dfe52ed84e655d3"
format_stamp: "Formatted at 2026-08-11 22:18:23 on dist-test-slave-jq4t"
server_key: "0b63af8ff5c370287eed6e39a1d69f0d"
server_key_iv: "a8a236588e182db3a2e3eb767d0ebf95"
server_key_version: "encryptionkey@0"
I20260811 22:18:23.050347 1243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "9a977e616b4c41bf8dfe52ed84e655d3"
format_stamp: "Formatted at 2026-08-11 22:18:23 on dist-test-slave-jq4t"
server_key: "0b63af8ff5c370287eed6e39a1d69f0d"
server_key_iv: "a8a236588e182db3a2e3eb767d0ebf95"
server_key_version: "encryptionkey@0"
I20260811 22:18:23.055883 1243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:18:23.059935 1271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:23.061691 1243 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:18:23.061908 1243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "9a977e616b4c41bf8dfe52ed84e655d3"
format_stamp: "Formatted at 2026-08-11 22:18:23 on dist-test-slave-jq4t"
server_key: "0b63af8ff5c370287eed6e39a1d69f0d"
server_key_iv: "a8a236588e182db3a2e3eb767d0ebf95"
server_key_version: "encryptionkey@0"
I20260811 22:18:23.062129 1243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:23.090112 1243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:23.094779 1243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:23.095099 1243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:23.098168 1243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:23.098312 1243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:23.098462 1243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:23.098562 1243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:18:23 dist-test-slave-jq4t krb5kdc[1052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486703, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:23.125146 1287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:23.098626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:43347 (local address 127.29.229.62:43861)
0811 22:18:23.099547 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:23.099552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:23.100283 (+ 731us) server_negotiation.cc:407] Connection header received
0811 22:18:23.102369 (+ 2086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:23.102375 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:23.102476 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:23.102630 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:23.104703 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:23.105804 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:23.108773 (+ 2969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:23.109396 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:23.109578 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:23.115993 (+ 6415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:23.116002 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:23.116008 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:23.116059 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:23.120825 (+ 4766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:23.121811 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:23.121821 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:23.121827 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:23.122042 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:23.122678 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:23.122688 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:23.122694 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:23.123204 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:23.123404 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:23.124295 (+ 891us) server_negotiation.cc:299] Negotiation successful
0811 22:18:23.124616 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":814,"thread_start_us":716,"threads_started":1}
I20260811 22:18:23.126650 1289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:23.099296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43861 (local address 127.29.229.1:43347)
0811 22:18:23.100131 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:23.100177 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:23.102082 (+ 1905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:23.102861 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:23.102869 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:23.103569 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:23.104408 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:23.104442 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:23.105982 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:23.105986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:23.108449 (+ 2463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:23.108513 (+ 64us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:23.109729 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:23.109737 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:23.110073 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:23.111123 (+ 1050us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:23.111167 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:23.115564 (+ 4397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:23.121077 (+ 5513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:23.121086 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:23.121109 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:23.121654 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:23.122296 (+ 642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:23.122300 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:23.122303 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:23.122535 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:23.123379 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:23.123387 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:23.123811 (+ 424us) client_negotiation.cc:769] Sending connection context
0811 22:18:23.124070 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 22:18:23.124859 (+ 789us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":342,"threads_started":1}
I20260811 22:18:23.154341 1287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:23.136025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:57647 (local address 127.29.229.62:43861)
0811 22:18:23.136287 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:23.136296 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:23.136805 (+ 509us) server_negotiation.cc:407] Connection header received
0811 22:18:23.137156 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:23.137166 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:23.137286 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:23.137452 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:23.138479 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:23.139935 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:23.141910 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:23.142556 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:23.142728 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:23.146732 (+ 4004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:23.146743 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:23.146751 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:23.146825 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:23.150631 (+ 3806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:23.151843 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:23.151850 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:23.151855 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:23.152062 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:23.152670 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:23.152690 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:23.152693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:23.153040 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:23.153234 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:23.153691 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 22:18:23.153942 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 22:18:23.154443 1411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:23.135966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43861 (local address 127.29.229.1:57647)
0811 22:18:23.136685 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:23.136714 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:23.136895 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:23.137513 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:23.137517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:23.138011 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:23.138314 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:23.138323 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:23.140149 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:23.140153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:23.141737 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:23.141756 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:23.142822 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:23.142829 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:23.143019 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:23.144184 (+ 1165us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:23.144286 (+ 102us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:23.146198 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:23.150893 (+ 4695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:23.150900 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:23.150905 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:23.151679 (+ 774us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:23.152217 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:23.152224 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:23.152229 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:23.152416 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:23.153192 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:23.153197 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:23.153408 (+ 211us) client_negotiation.cc:769] Sending connection context
0811 22:18:23.153568 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 22:18:23.153971 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":392,"threads_started":1}
I20260811 22:18:23.165892 1243 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:35515
I20260811 22:18:23.166045 1491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:35515 every 8 connection(s)
I20260811 22:18:23.167313 1243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:18:23.170187 1496 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43861
I20260811 22:18:23.170493 1496 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:23.171252 1496 heartbeater.cc:511] Master 127.29.229.62:43861 requested a full tablet report, sending...
I20260811 22:18:23.173902 1124 ts_manager.cc:194] Registered new tserver with Master: 9a977e616b4c41bf8dfe52ed84e655d3 (127.29.229.1:35515)
I20260811 22:18:23.175630 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 1243
I20260811 22:18:23.175616 1124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:43347
I20260811 22:18:23.176008 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:18:23.177160 30612 external_mini_cluster.cc:1428] Setting key 214985a5dfe95a0254c744138bfcb527
WARNING: no policy specified for kudu/127.29.229.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
I20260811 22:18:23.236931 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43861
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:38689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:23.613663 1505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:23.614024 1505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:23.614215 1505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:23.621960 1505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:23.622114 1505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:23.622259 1505 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:18:23.622349 1505 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:18:23.622484 1505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:18:23.632848 1505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:38689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.229.62:43861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:23.634220 1505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:23.635557 1505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:23.650605 1516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:23.650669 1514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:23.650727 1518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:23.651966 1505 server_base.cc:1034] running on GCE node
I20260811 22:18:23.652729 1505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:23.654290 1505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:23.655665 1505 hybrid_clock.cc:648] HybridClock initialized: now 1786486703655580 us; error 67 us; skew 500 ppm
I20260811 22:18:23.656066 1505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:23 dist-test-slave-jq4t krb5kdc[1052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486703, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:23.660048 1505 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:18:23.661823 1505 webserver.cc:460] Webserver started at http://127.29.229.2:44205/ using document root <none> and password file <none>
I20260811 22:18:23.662618 1505 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:23.662732 1505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:23.663015 1505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:23.666158 1505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "3d16b93155a24aefa9a585715d61d320"
format_stamp: "Formatted at 2026-08-11 22:18:23 on dist-test-slave-jq4t"
server_key: "479027117894aef9773d51c26ee8796e"
server_key_iv: "770334ed54e48d5d5a07f5c5bb89fe05"
server_key_version: "encryptionkey@0"
I20260811 22:18:23.666992 1505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "3d16b93155a24aefa9a585715d61d320"
format_stamp: "Formatted at 2026-08-11 22:18:23 on dist-test-slave-jq4t"
server_key: "479027117894aef9773d51c26ee8796e"
server_key_iv: "770334ed54e48d5d5a07f5c5bb89fe05"
server_key_version: "encryptionkey@0"
I20260811 22:18:23.672999 1505 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260811 22:18:23.677270 1533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:23.679005 1505 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:18:23.679164 1505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "3d16b93155a24aefa9a585715d61d320"
format_stamp: "Formatted at 2026-08-11 22:18:23 on dist-test-slave-jq4t"
server_key: "479027117894aef9773d51c26ee8796e"
server_key_iv: "770334ed54e48d5d5a07f5c5bb89fe05"
server_key_version: "encryptionkey@0"
I20260811 22:18:23.679358 1505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:23.726172 1505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:23.731556 1505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:23.731904 1505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:23.735167 1505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:23.735275 1505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:23.735456 1505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:23.735540 1505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:18:23 dist-test-slave-jq4t krb5kdc[1052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486703, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:23.768390 1547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:23.736645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:52011 (local address 127.29.229.62:43861)
0811 22:18:23.737130 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:23.737137 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:23.738487 (+ 1350us) server_negotiation.cc:407] Connection header received
0811 22:18:23.741983 (+ 3496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:23.741992 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:23.742167 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:23.742336 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:23.745974 (+ 3638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:23.747129 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:23.750968 (+ 3839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:23.751636 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:23.751802 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:23.758240 (+ 6438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:23.758251 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:23.758259 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:23.758316 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:23.763555 (+ 5239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:23.764743 (+ 1188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:23.764753 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:23.764759 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:23.765079 (+ 320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:23.765737 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:23.765746 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:23.765753 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:23.766265 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:23.766514 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:23.767436 (+ 922us) server_negotiation.cc:299] Negotiation successful
0811 22:18:23.767795 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":233,"threads_started":1}
I20260811 22:18:23.768926 1549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:23.737328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43861 (local address 127.29.229.2:52011)
0811 22:18:23.738283 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:23.738341 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:23.741289 (+ 2948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:23.742830 (+ 1541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:23.742851 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:23.744248 (+ 1397us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:23.745551 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:23.745587 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:23.747390 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:23.747399 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:23.750671 (+ 3272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:23.750704 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:23.751984 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:23.751997 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:23.752566 (+ 569us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:23.753655 (+ 1089us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:23.753697 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:23.757851 (+ 4154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:23.763808 (+ 5957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:23.763823 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:23.763855 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:23.764580 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:23.765268 (+ 688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:23.765272 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:23.765274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:23.765595 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:23.766439 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:23.766447 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:23.766928 (+ 481us) client_negotiation.cc:769] Sending connection context
0811 22:18:23.767200 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 22:18:23.767646 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":419,"threads_started":1}
I20260811 22:18:23.793435 1547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:23.775718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:55239 (local address 127.29.229.62:43861)
0811 22:18:23.775989 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:23.775999 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:23.776955 (+ 956us) server_negotiation.cc:407] Connection header received
0811 22:18:23.777190 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:23.777196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:23.777280 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:23.777397 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:23.778933 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:23.780057 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:23.781788 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:23.782365 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:23.782495 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:23.785136 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:23.785149 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:23.785153 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:23.785187 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:23.789940 (+ 4753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:23.791018 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:23.791024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:23.791029 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:23.791194 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:23.791626 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:23.791632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:23.791636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:23.791992 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:23.792160 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:23.792777 (+ 617us) server_negotiation.cc:299] Negotiation successful
0811 22:18:23.793048 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:18:23.793512 1667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:23.775596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43861 (local address 127.29.229.2:55239)
0811 22:18:23.776813 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:23.776850 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:23.777045 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:23.777418 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:23.777427 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:23.778194 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:23.778678 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:23.778698 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:23.780202 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:23.780208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:23.781634 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:23.781649 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:23.782806 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:23.782812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:23.782949 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:23.783760 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:23.783780 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:23.784889 (+ 1109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:23.790172 (+ 5283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:23.790178 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:23.790181 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:23.790810 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:23.791320 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:23.791325 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:23.791329 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:23.791487 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:23.792121 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:23.792126 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:23.792390 (+ 264us) client_negotiation.cc:769] Sending connection context
0811 22:18:23.792602 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 22:18:23.792942 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":952,"thread_start_us":607,"threads_started":1}
I20260811 22:18:23.795940 1505 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:40199
I20260811 22:18:23.796051 1731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:40199 every 8 connection(s)
I20260811 22:18:23.797485 1505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:18:23.800115 1734 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43861
I20260811 22:18:23.800345 1734 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:23.801112 1734 heartbeater.cc:511] Master 127.29.229.62:43861 requested a full tablet report, sending...
I20260811 22:18:23.802994 1124 ts_manager.cc:194] Registered new tserver with Master: 3d16b93155a24aefa9a585715d61d320 (127.29.229.2:40199)
I20260811 22:18:23.804162 1124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:52011
I20260811 22:18:23.805653 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 1505
I20260811 22:18:23.805958 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:18:23.806439 30612 external_mini_cluster.cc:1428] Setting key 6dba0d3b52be84d35d177be844c25344
WARNING: no policy specified for kudu/127.29.229.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
I20260811 22:18:23.865216 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43861
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:38689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:18:24.179205 1496 heartbeater.cc:503] Master 127.29.229.62:43861 was elected leader, sending a full tablet report...
W20260811 22:18:24.246949 1742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:24.247251 1742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:24.247452 1742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:24.255087 1742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:24.255226 1742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:24.255415 1742 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:18:24.255511 1742 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:18:24.255646 1742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:18:24.267843 1742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:38689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.229.62:43861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:24.269348 1742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:24.271121 1742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:24.285701 1760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:24.285864 1756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:24.285869 1755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:24.290735 1742 server_base.cc:1034] running on GCE node
I20260811 22:18:24.291850 1742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:24.293372 1742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:24.294716 1742 hybrid_clock.cc:648] HybridClock initialized: now 1786486704294669 us; error 65 us; skew 500 ppm
I20260811 22:18:24.295130 1742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:24 dist-test-slave-jq4t krb5kdc[1052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486704, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:24.299649 1742 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:18:24.301921 1742 webserver.cc:460] Webserver started at http://127.29.229.3:43475/ using document root <none> and password file <none>
I20260811 22:18:24.302996 1742 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:24.303197 1742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:24.303570 1742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:24.306921 1742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "84bfbf7596a846f0a69c8c26259f7685"
format_stamp: "Formatted at 2026-08-11 22:18:24 on dist-test-slave-jq4t"
server_key: "aa8586ecdbab7a845d740d42809a8af5"
server_key_iv: "3059e7d60c8d441fad5f8d10fce5a317"
server_key_version: "encryptionkey@0"
I20260811 22:18:24.307750 1742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "84bfbf7596a846f0a69c8c26259f7685"
format_stamp: "Formatted at 2026-08-11 22:18:24 on dist-test-slave-jq4t"
server_key: "aa8586ecdbab7a845d740d42809a8af5"
server_key_iv: "3059e7d60c8d441fad5f8d10fce5a317"
server_key_version: "encryptionkey@0"
I20260811 22:18:24.313762 1742 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 22:18:24.318886 1767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:24.320722 1742 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 22:18:24.320940 1742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "84bfbf7596a846f0a69c8c26259f7685"
format_stamp: "Formatted at 2026-08-11 22:18:24 on dist-test-slave-jq4t"
server_key: "aa8586ecdbab7a845d740d42809a8af5"
server_key_iv: "3059e7d60c8d441fad5f8d10fce5a317"
server_key_version: "encryptionkey@0"
I20260811 22:18:24.321198 1742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:24.347486 1742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:24.352943 1742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:24.353293 1742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:24.356351 1742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:24.356537 1742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:24.356695 1742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:24.356802 1742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:18:24 dist-test-slave-jq4t krb5kdc[1052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486704, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:24.386059 1781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.357559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:56301 (local address 127.29.229.62:43861)
0811 22:18:24.358107 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:24.358115 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:24.360150 (+ 2035us) server_negotiation.cc:407] Connection header received
0811 22:18:24.363255 (+ 3105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:24.363264 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:24.363390 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:24.363542 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:24.366016 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.367477 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.369719 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.370252 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.370406 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.376153 (+ 5747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:24.376165 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:24.376173 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:24.376240 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:24.381108 (+ 4868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:24.382335 (+ 1227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:24.382342 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:24.382345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:24.382527 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:24.383194 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:24.383200 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:24.383203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:24.383547 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:24.383733 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:24.385389 (+ 1656us) server_negotiation.cc:299] Negotiation successful
0811 22:18:24.385652 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":211,"threads_started":1}
I20260811 22:18:24.387259 1782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.358554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43861 (local address 127.29.229.3:56301)
0811 22:18:24.359900 (+ 1346us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:24.359985 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:24.362837 (+ 2852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:24.363857 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:24.363874 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:24.364832 (+ 958us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:24.365756 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.365812 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.367675 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.367679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.369437 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.369451 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.370837 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.370844 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.371216 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.372069 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:24.372100 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:24.375666 (+ 3566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:24.381311 (+ 5645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:24.381336 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:24.381365 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:24.382147 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:24.382705 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:24.382711 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:24.382716 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:24.383056 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:24.383703 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:24.383712 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:24.384882 (+ 1170us) client_negotiation.cc:769] Sending connection context
0811 22:18:24.385280 (+ 398us) client_negotiation.cc:240] Negotiation successful
0811 22:18:24.385938 (+ 658us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":918,"thread_start_us":432,"threads_started":1}
I20260811 22:18:24.415290 1889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.395200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43861 (local address 127.29.229.3:43467)
0811 22:18:24.395905 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:24.395925 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:24.396302 (+ 377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:24.397649 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:24.397654 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:24.398828 (+ 1174us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:24.399442 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.399468 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.401241 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.401252 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.403391 (+ 2139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.403412 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.404736 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.404743 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.404914 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.405731 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:24.405750 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:24.406612 (+ 862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:24.411391 (+ 4779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:24.411398 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:24.411404 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:24.412139 (+ 735us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:24.412692 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:24.412697 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:24.412700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:24.412859 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:24.413583 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:24.413592 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:24.413906 (+ 314us) client_negotiation.cc:769] Sending connection context
0811 22:18:24.414113 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 22:18:24.414548 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":412,"threads_started":1}
I20260811 22:18:24.415577 1781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.394772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:43467 (local address 127.29.229.62:43861)
0811 22:18:24.394985 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:24.394992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:24.396050 (+ 1058us) server_negotiation.cc:407] Connection header received
0811 22:18:24.396854 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:24.396860 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:24.397002 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:24.397073 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:24.399680 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.401017 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.403623 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.404450 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.404663 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.406874 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:24.406884 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:24.406892 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:24.406947 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:24.411160 (+ 4213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:24.412302 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:24.412311 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:24.412317 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:24.412544 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:24.412991 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:24.412999 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:24.413003 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:24.413456 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:24.413618 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:24.414974 (+ 1356us) server_negotiation.cc:299] Negotiation successful
0811 22:18:24.415270 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:18:24.424350 1742 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:34577
I20260811 22:18:24.424518 1972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:34577 every 8 connection(s)
I20260811 22:18:24.425803 1742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:18:24.428350 1974 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43861
I20260811 22:18:24.428628 1974 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:24.430056 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 1742
I20260811 22:18:24.430205 1974 heartbeater.cc:511] Master 127.29.229.62:43861 requested a full tablet report, sending...
I20260811 22:18:24.430310 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:18:24.430920 30612 external_mini_cluster.cc:1428] Setting key 80afacc6f18150ae775e2768aab0a0df
I20260811 22:18:24.431918 1124 ts_manager.cc:194] Registered new tserver with Master: 84bfbf7596a846f0a69c8c26259f7685 (127.29.229.3:34577)
I20260811 22:18:24.433068 1124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:56301
I20260811 22:18:24.435407 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:18:24 dist-test-slave-jq4t krb5kdc[1052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:18:24 dist-test-slave-jq4t krb5kdc[1052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:24.499080 1781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.481632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47456 (local address 127.29.229.62:43861)
0811 22:18:24.481828 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:24.481837 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:24.482331 (+ 494us) server_negotiation.cc:407] Connection header received
0811 22:18:24.482471 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:24.482475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:24.482544 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:24.482646 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:24.485144 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.485898 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.487293 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.487963 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.488078 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.492898 (+ 4820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:24.492904 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:24.492908 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:24.492963 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:24.495863 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:24.496673 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:24.496678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:24.496681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:24.496859 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:24.497402 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:24.497407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:24.497410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:24.497761 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:24.497999 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:24.498520 (+ 521us) server_negotiation.cc:299] Negotiation successful
0811 22:18:24.498719 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:18:24.504194 1124 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:18:24.511399 1124 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:18:24.551290 1997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.526129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:35515 (local address 127.0.0.1:58736)
0811 22:18:24.536739 (+ 10610us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:24.536818 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:24.537076 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:24.538435 (+ 1359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:24.538447 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:24.538492 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:24.539267 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.539288 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.542668 (+ 3380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.542676 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.546355 (+ 3679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.546379 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.549734 (+ 3355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.549744 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.550098 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.550173 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 22:18:24.550343 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 22:18:24.550628 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10291,"mutex_wait_us":44,"thread_start_us":11125,"threads_started":1}
I20260811 22:18:24.552655 1995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.526291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:40199 (local address 127.0.0.1:57330)
0811 22:18:24.537224 (+ 10933us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:24.537242 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:24.537365 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:24.538477 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:24.538482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:24.538509 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:24.539280 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.539301 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.541935 (+ 2634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.541944 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.546768 (+ 4824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.546791 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.551375 (+ 4584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.551385 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.551672 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.551725 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 22:18:24.551839 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 22:18:24.552033 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10798,"mutex_wait_us":34,"thread_start_us":10010,"threads_started":1}
I20260811 22:18:24.552667 1998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.526360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58736 (local address 127.29.229.1:35515)
0811 22:18:24.536754 (+ 10394us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:24.536768 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:24.537493 (+ 725us) server_negotiation.cc:407] Connection header received
0811 22:18:24.537613 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:24.537622 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:24.537908 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:24.538112 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:24.539533 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.542402 (+ 2869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.546627 (+ 4225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.549036 (+ 2409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.549198 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.549482 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:24.551948 (+ 2466us) server_negotiation.cc:299] Negotiation successful
0811 22:18:24.552224 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9886,"thread_start_us":10159,"threads_started":1}
I20260811 22:18:24.553608 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.526930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57330 (local address 127.29.229.2:40199)
0811 22:18:24.537657 (+ 10727us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:24.537666 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:24.537696 (+ 30us) server_negotiation.cc:407] Connection header received
0811 22:18:24.537794 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:24.537801 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:24.538058 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:24.538350 (+ 292us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:24.539569 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.541705 (+ 2136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.547069 (+ 5364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.550692 (+ 3623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.550899 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.551155 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:24.552937 (+ 1782us) server_negotiation.cc:299] Negotiation successful
0811 22:18:24.553162 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10535,"thread_start_us":11253,"threads_started":1}
I20260811 22:18:24.558674 1611 tablet_service.cc:1467] Processing CreateTablet for tablet 1c276dacabec46fea1e86d6171018d04 (DEFAULT_TABLE table=test-table [id=2395c836b28f4e5ca8f56c4262063a16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:24.558986 1360 tablet_service.cc:1467] Processing CreateTablet for tablet 1c276dacabec46fea1e86d6171018d04 (DEFAULT_TABLE table=test-table [id=2395c836b28f4e5ca8f56c4262063a16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:24.560201 1611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c276dacabec46fea1e86d6171018d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:24.561206 1360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c276dacabec46fea1e86d6171018d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:24.562541 1997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.538701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:34577 (local address 127.0.0.1:51788)
0811 22:18:24.551575 (+ 12874us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:24.551594 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:24.551711 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:24.554231 (+ 2520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:24.554237 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:24.554265 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:24.554475 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.554486 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.556554 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.556559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.558921 (+ 2362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.558940 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.561636 (+ 2696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.561647 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.561860 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.561898 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 22:18:24.562021 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 22:18:24.562223 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12770,"thread_start_us":14843,"threads_started":1}
I20260811 22:18:24.562637 2003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.538083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51788 (local address 127.29.229.3:34577)
0811 22:18:24.553554 (+ 15471us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:24.553566 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:24.553609 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:18:24.553727 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:24.553738 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:24.554031 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:24.554247 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:24.554692 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.556354 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.559159 (+ 2805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.561417 (+ 2258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.561572 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.561835 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:24.562073 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 22:18:24.562279 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15266,"thread_start_us":15045,"threads_started":1}
I20260811 22:18:24.567945 1862 tablet_service.cc:1467] Processing CreateTablet for tablet 1c276dacabec46fea1e86d6171018d04 (DEFAULT_TABLE table=test-table [id=2395c836b28f4e5ca8f56c4262063a16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:24.569660 1862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c276dacabec46fea1e86d6171018d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:24.574738 2013 tablet_bootstrap.cc:492] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320: Bootstrap starting.
I20260811 22:18:24.577040 2014 tablet_bootstrap.cc:492] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3: Bootstrap starting.
I20260811 22:18:24.580626 2014 tablet_bootstrap.cc:654] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:24.581125 2013 tablet_bootstrap.cc:654] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:24.581804 2014 log.cc:824] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:24.582480 2013 log.cc:824] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:24.585543 2014 tablet_bootstrap.cc:492] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3: No bootstrap required, opened a new log
I20260811 22:18:24.585857 2014 ts_tablet_manager.cc:1397] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260811 22:18:24.586748 2013 tablet_bootstrap.cc:492] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320: No bootstrap required, opened a new log
I20260811 22:18:24.586737 2020 tablet_bootstrap.cc:492] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685: Bootstrap starting.
I20260811 22:18:24.587024 2013 ts_tablet_manager.cc:1397] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.006s
I20260811 22:18:24.590502 2020 tablet_bootstrap.cc:654] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:24.591794 2020 log.cc:824] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:24.595423 2020 tablet_bootstrap.cc:492] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685: No bootstrap required, opened a new log
I20260811 22:18:24.595713 2020 ts_tablet_manager.cc:1397] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260811 22:18:24.597048 2014 raft_consensus.cc:348] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:24.597507 2014 raft_consensus.cc:374] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:24.597620 2014 raft_consensus.cc:729] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a977e616b4c41bf8dfe52ed84e655d3, State: Initialized, Role: FOLLOWER
I20260811 22:18:24.597801 2013 raft_consensus.cc:348] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:24.598235 2013 raft_consensus.cc:374] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:24.598351 2013 raft_consensus.cc:729] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d16b93155a24aefa9a585715d61d320, State: Initialized, Role: FOLLOWER
I20260811 22:18:24.598302 2014 consensus_queue.cc:260] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:24.599074 2013 consensus_queue.cc:260] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:24.600191 2014 ts_tablet_manager.cc:1428] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260811 22:18:24.601573 1734 heartbeater.cc:503] Master 127.29.229.62:43861 was elected leader, sending a full tablet report...
I20260811 22:18:24.602222 2013 ts_tablet_manager.cc:1428] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260811 22:18:24.605908 2020 raft_consensus.cc:348] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:24.606393 2020 raft_consensus.cc:374] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:24.606495 2020 raft_consensus.cc:729] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84bfbf7596a846f0a69c8c26259f7685, State: Initialized, Role: FOLLOWER
I20260811 22:18:24.607240 2020 consensus_queue.cc:260] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:24.609197 1974 heartbeater.cc:503] Master 127.29.229.62:43861 was elected leader, sending a full tablet report...
I20260811 22:18:24.609715 2020 ts_tablet_manager.cc:1428] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685: Time spent starting tablet: real 0.014s user 0.012s sys 0.003s
I20260811 22:18:24.616344 2026 raft_consensus.cc:482] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:18:24.616690 2026 raft_consensus.cc:504] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:24.618824 2026 leader_election.cc:288] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a977e616b4c41bf8dfe52ed84e655d3 (127.29.229.1:35515), 3d16b93155a24aefa9a585715d61d320 (127.29.229.2:40199)
I20260811 22:18:24.625206 1782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.619192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:35515 (local address 127.29.229.3:43645)
0811 22:18:24.619454 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:24.619476 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:24.619617 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:24.620080 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:24.620086 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:24.620113 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:24.620347 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.620359 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.621937 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.621942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.623277 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.623288 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.624378 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.624384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.624644 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.624669 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:18:24.624777 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 22:18:24.624914 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 22:18:24.625203 1998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.619528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:43645 (local address 127.29.229.1:35515)
0811 22:18:24.619758 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:24.619767 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:24.619785 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:18:24.619859 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:24.619863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:24.619932 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:24.620049 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:24.620954 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.621799 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.623441 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.624222 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.624393 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.624470 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:24.624818 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 22:18:24.624955 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 22:18:24.625660 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.619373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:53537 (local address 127.29.229.2:40199)
0811 22:18:24.619612 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:24.619621 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:24.620358 (+ 737us) server_negotiation.cc:407] Connection header received
0811 22:18:24.620541 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:24.620549 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:24.620642 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:24.620742 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:24.621359 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.622091 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.623770 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.624663 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.624863 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.624949 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:24.625275 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 22:18:24.625404 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:18:24.625947 2027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.619356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:40199 (local address 127.29.229.3:53537)
0811 22:18:24.620244 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:24.620262 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:24.620378 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:24.620780 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:24.620783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:24.620805 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:24.621079 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.621087 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.622236 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.622241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.623614 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.623626 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.624941 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.624946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.625091 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.625112 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:18:24.625230 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 22:18:24.625364 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"thread_start_us":554,"threads_started":1}
I20260811 22:18:24.626412 1414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c276dacabec46fea1e86d6171018d04" candidate_uuid: "84bfbf7596a846f0a69c8c26259f7685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" is_pre_election: true
I20260811 22:18:24.627036 1414 raft_consensus.cc:2454] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84bfbf7596a846f0a69c8c26259f7685 in term 0.
I20260811 22:18:24.627007 1651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c276dacabec46fea1e86d6171018d04" candidate_uuid: "84bfbf7596a846f0a69c8c26259f7685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d16b93155a24aefa9a585715d61d320" is_pre_election: true
I20260811 22:18:24.627568 1651 raft_consensus.cc:2454] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84bfbf7596a846f0a69c8c26259f7685 in term 0.
I20260811 22:18:24.627748 1772 leader_election.cc:302] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84bfbf7596a846f0a69c8c26259f7685, 9a977e616b4c41bf8dfe52ed84e655d3; no voters:
I20260811 22:18:24.628207 2026 raft_consensus.cc:2781] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:18:24.628376 2026 raft_consensus.cc:482] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:18:24.628525 2026 raft_consensus.cc:3037] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:24.631029 2026 raft_consensus.cc:504] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:24.631721 2026 leader_election.cc:288] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [CANDIDATE]: Term 1 election: Requested vote from peers 9a977e616b4c41bf8dfe52ed84e655d3 (127.29.229.1:35515), 3d16b93155a24aefa9a585715d61d320 (127.29.229.2:40199)
I20260811 22:18:24.632138 1414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c276dacabec46fea1e86d6171018d04" candidate_uuid: "84bfbf7596a846f0a69c8c26259f7685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a977e616b4c41bf8dfe52ed84e655d3"
I20260811 22:18:24.632264 1651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c276dacabec46fea1e86d6171018d04" candidate_uuid: "84bfbf7596a846f0a69c8c26259f7685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d16b93155a24aefa9a585715d61d320"
I20260811 22:18:24.632370 1414 raft_consensus.cc:3037] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:24.632470 1651 raft_consensus.cc:3037] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:24.635731 1414 raft_consensus.cc:2454] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84bfbf7596a846f0a69c8c26259f7685 in term 1.
I20260811 22:18:24.635731 1651 raft_consensus.cc:2454] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84bfbf7596a846f0a69c8c26259f7685 in term 1.
I20260811 22:18:24.636216 1772 leader_election.cc:302] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84bfbf7596a846f0a69c8c26259f7685, 9a977e616b4c41bf8dfe52ed84e655d3; no voters:
I20260811 22:18:24.636565 2026 raft_consensus.cc:2781] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:24.637197 2026 raft_consensus.cc:686] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [term 1 LEADER]: Becoming Leader. State: Replica: 84bfbf7596a846f0a69c8c26259f7685, State: Running, Role: LEADER
I20260811 22:18:24.637714 2026 consensus_queue.cc:237] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:24.642354 1123 catalog_manager.cc:5495] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84bfbf7596a846f0a69c8c26259f7685 (127.29.229.3). New cstate: current_term: 1 leader_uuid: "84bfbf7596a846f0a69c8c26259f7685" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } health_report { overall_health: HEALTHY } } }
W20260811 22:18:24.670303 1501 tablet.cc:2369] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:18:24.677740 1975 tablet.cc:2369] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 22:18:24 dist-test-slave-jq4t krb5kdc[1052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486702, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:24.695268 2040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.679771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43861 (local address 127.0.0.1:47458)
0811 22:18:24.680811 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:24.680843 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:24.681049 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:24.681504 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:24.681509 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:24.682331 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:24.682616 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.682626 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.683840 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.683846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.685067 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.685080 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.685847 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.685852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.685990 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.686682 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:24.686705 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:24.689713 (+ 3008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:24.692871 (+ 3158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:24.692882 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:24.692888 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:24.693329 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:24.693693 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:24.693696 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:24.693698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:24.693833 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:24.694382 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:24.694389 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:24.694547 (+ 158us) client_negotiation.cc:769] Sending connection context
0811 22:18:24.694677 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 22:18:24.694910 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"thread_start_us":477,"threads_started":1}
I20260811 22:18:24.695369 1781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.679707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47458 (local address 127.29.229.62:43861)
0811 22:18:24.679930 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:24.679938 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:24.681023 (+ 1085us) server_negotiation.cc:407] Connection header received
0811 22:18:24.681221 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:24.681225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:24.681300 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:24.681408 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:24.682815 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.683620 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.685217 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.685656 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.685764 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.689956 (+ 4192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:24.689963 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:24.689971 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:24.690005 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:24.692726 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:24.693453 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:24.693458 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:24.693460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:24.693593 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:24.693964 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:24.693968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:24.693970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:24.694255 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:24.694384 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:24.694779 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 22:18:24.695043 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":71}
I20260811 22:18:24.701490 1123 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:47458:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:18:24.702780 1123 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:18:24.710834 1862 tablet_service.cc:1467] Processing CreateTablet for tablet 0d2e0e0cafd444dab3fc32c4ced843f7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=384010df3b584f8ca7ef9625e84f7bfd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:18:24.710362 1611 tablet_service.cc:1467] Processing CreateTablet for tablet 0d2e0e0cafd444dab3fc32c4ced843f7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=384010df3b584f8ca7ef9625e84f7bfd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:18:24.710358 1360 tablet_service.cc:1467] Processing CreateTablet for tablet 0d2e0e0cafd444dab3fc32c4ced843f7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=384010df3b584f8ca7ef9625e84f7bfd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:18:24.711781 1862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d2e0e0cafd444dab3fc32c4ced843f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:24.711869 1360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d2e0e0cafd444dab3fc32c4ced843f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:24.711865 1611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d2e0e0cafd444dab3fc32c4ced843f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:24.718241 2020 tablet_bootstrap.cc:492] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685: Bootstrap starting.
I20260811 22:18:24.719738 2014 tablet_bootstrap.cc:492] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 9a977e616b4c41bf8dfe52ed84e655d3: Bootstrap starting.
I20260811 22:18:24.719936 2013 tablet_bootstrap.cc:492] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320: Bootstrap starting.
I20260811 22:18:24.720244 2020 tablet_bootstrap.cc:654] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:24.721745 2013 tablet_bootstrap.cc:654] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:24.721789 2014 tablet_bootstrap.cc:654] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 9a977e616b4c41bf8dfe52ed84e655d3: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:24.723193 2020 tablet_bootstrap.cc:492] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685: No bootstrap required, opened a new log
I20260811 22:18:24.723376 2020 ts_tablet_manager.cc:1397] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20260811 22:18:24.724206 2014 tablet_bootstrap.cc:492] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 9a977e616b4c41bf8dfe52ed84e655d3: No bootstrap required, opened a new log
I20260811 22:18:24.724275 2020 raft_consensus.cc:348] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:24.724447 2014 ts_tablet_manager.cc:1397] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 9a977e616b4c41bf8dfe52ed84e655d3: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:18:24.724558 2020 raft_consensus.cc:374] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:24.724714 2020 raft_consensus.cc:729] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84bfbf7596a846f0a69c8c26259f7685, State: Initialized, Role: FOLLOWER
I20260811 22:18:24.724993 2020 consensus_queue.cc:260] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:24.725579 2013 tablet_bootstrap.cc:492] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320: No bootstrap required, opened a new log
I20260811 22:18:24.725441 2026 tablet_replica.cc:439] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } } }
I20260811 22:18:24.725739 2013 ts_tablet_manager.cc:1397] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:18:24.725732 2026 tablet_replica.cc:442] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:18:24.726306 2020 ts_tablet_manager.cc:1428] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:18:24.726326 2014 raft_consensus.cc:348] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:24.726624 2014 raft_consensus.cc:374] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:24.726745 2014 raft_consensus.cc:729] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a977e616b4c41bf8dfe52ed84e655d3, State: Initialized, Role: FOLLOWER
I20260811 22:18:24.726562 2013 raft_consensus.cc:348] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:24.726861 2013 raft_consensus.cc:374] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:24.726985 2013 raft_consensus.cc:729] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d16b93155a24aefa9a585715d61d320, State: Initialized, Role: FOLLOWER
I20260811 22:18:24.727013 2014 consensus_queue.cc:260] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 9a977e616b4c41bf8dfe52ed84e655d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:24.727368 2013 consensus_queue.cc:260] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:24.727521 2024 tablet_replica.cc:439] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 9a977e616b4c41bf8dfe52ed84e655d3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } } }
I20260811 22:18:24.727893 2024 tablet_replica.cc:442] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 9a977e616b4c41bf8dfe52ed84e655d3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:18:24.728032 2013 ts_tablet_manager.cc:1428] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 22:18:24.728160 2025 tablet_replica.cc:439] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } } }
I20260811 22:18:24.728515 2025 tablet_replica.cc:442] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:18:24.729872 2014 ts_tablet_manager.cc:1428] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 9a977e616b4c41bf8dfe52ed84e655d3: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 22:18:24.733623 2026 raft_consensus.cc:482] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:18:24.733839 2026 raft_consensus.cc:504] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:24.734444 2026 leader_election.cc:288] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a977e616b4c41bf8dfe52ed84e655d3 (127.29.229.1:35515), 3d16b93155a24aefa9a585715d61d320 (127.29.229.2:40199)
I20260811 22:18:24.735008 1651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d2e0e0cafd444dab3fc32c4ced843f7" candidate_uuid: "84bfbf7596a846f0a69c8c26259f7685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d16b93155a24aefa9a585715d61d320" is_pre_election: true
I20260811 22:18:24.735005 1414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d2e0e0cafd444dab3fc32c4ced843f7" candidate_uuid: "84bfbf7596a846f0a69c8c26259f7685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" is_pre_election: true
I20260811 22:18:24.735278 1651 raft_consensus.cc:2454] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84bfbf7596a846f0a69c8c26259f7685 in term 0.
I20260811 22:18:24.735277 1414 raft_consensus.cc:2454] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84bfbf7596a846f0a69c8c26259f7685 in term 0.
I20260811 22:18:24.735754 1772 leader_election.cc:302] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3d16b93155a24aefa9a585715d61d320, 84bfbf7596a846f0a69c8c26259f7685, 9a977e616b4c41bf8dfe52ed84e655d3; no voters:
I20260811 22:18:24.736157 2026 raft_consensus.cc:2781] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:18:24.736285 2026 raft_consensus.cc:482] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:18:24.736419 2026 raft_consensus.cc:3037] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:24.738847 2026 raft_consensus.cc:504] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:24.739434 2026 leader_election.cc:288] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 [CANDIDATE]: Term 1 election: Requested vote from peers 9a977e616b4c41bf8dfe52ed84e655d3 (127.29.229.1:35515), 3d16b93155a24aefa9a585715d61d320 (127.29.229.2:40199)
I20260811 22:18:24.739975 1651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d2e0e0cafd444dab3fc32c4ced843f7" candidate_uuid: "84bfbf7596a846f0a69c8c26259f7685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d16b93155a24aefa9a585715d61d320"
I20260811 22:18:24.740242 1651 raft_consensus.cc:3037] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:24.740710 1414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d2e0e0cafd444dab3fc32c4ced843f7" candidate_uuid: "84bfbf7596a846f0a69c8c26259f7685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a977e616b4c41bf8dfe52ed84e655d3"
I20260811 22:18:24.740988 1414 raft_consensus.cc:3037] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:24.743525 1651 raft_consensus.cc:2454] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84bfbf7596a846f0a69c8c26259f7685 in term 1.
I20260811 22:18:24.744051 1774 leader_election.cc:302] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d16b93155a24aefa9a585715d61d320, 84bfbf7596a846f0a69c8c26259f7685; no voters:
I20260811 22:18:24.744320 1414 raft_consensus.cc:2454] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84bfbf7596a846f0a69c8c26259f7685 in term 1.
I20260811 22:18:24.744670 2026 raft_consensus.cc:2781] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:24.744949 2026 raft_consensus.cc:686] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 [term 1 LEADER]: Becoming Leader. State: Replica: 84bfbf7596a846f0a69c8c26259f7685, State: Running, Role: LEADER
I20260811 22:18:24.745242 2026 consensus_queue.cc:237] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:24.747237 2028 tablet_replica.cc:439] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685: TxnStatusTablet state changed. Reason: New leader 84bfbf7596a846f0a69c8c26259f7685. Latest consensus state: current_term: 1 leader_uuid: "84bfbf7596a846f0a69c8c26259f7685" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } } }
I20260811 22:18:24.747733 2028 tablet_replica.cc:442] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:18:24.748591 2048 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:18:24.748905 2048 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:18:24.749202 1122 catalog_manager.cc:5495] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84bfbf7596a846f0a69c8c26259f7685 (127.29.229.3). New cstate: current_term: 1 leader_uuid: "84bfbf7596a846f0a69c8c26259f7685" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } health_report { overall_health: HEALTHY } } }
I20260811 22:18:24.773721 2040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.764415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:34577 (local address 127.0.0.1:51796)
0811 22:18:24.764735 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:24.764761 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:24.764917 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:24.765570 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:24.765575 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:24.765591 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:18:24.765847 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.765861 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.767508 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.767517 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.769255 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.769274 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.770607 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.770615 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.771141 (+ 526us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.771281 (+ 140us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:18:24.772938 (+ 1657us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:18:24.772991 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 22:18:24.773100 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 22:18:24.773311 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 22:18:24.773728 2003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.764763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51796 (local address 127.29.229.3:34577)
0811 22:18:24.765016 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:24.765022 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:24.765046 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:18:24.765198 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:24.765202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:24.765312 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:24.765491 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:18:24.766211 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.767311 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.769504 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.770304 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.770448 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.771666 (+ 1218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:18:24.772774 (+ 1108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:18:24.772958 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:24.773155 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 22:18:24.773288 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:18:24.784264 1651 raft_consensus.cc:1261] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320 [term 1 FOLLOWER]: Refusing update from remote peer 84bfbf7596a846f0a69c8c26259f7685: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:24.784396 1414 raft_consensus.cc:1261] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 FOLLOWER]: Refusing update from remote peer 84bfbf7596a846f0a69c8c26259f7685: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:24.786207 2028 consensus_queue.cc:1035] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:24.787052 2028 consensus_queue.cc:1035] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:18:24.799881 1735 tablet.cc:2369] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:18:24.813863 2025 tablet_replica.cc:439] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320: TxnStatusTablet state changed. Reason: New leader 84bfbf7596a846f0a69c8c26259f7685. Latest consensus state: current_term: 1 leader_uuid: "84bfbf7596a846f0a69c8c26259f7685" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } } }
I20260811 22:18:24.814276 2025 tablet_replica.cc:442] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:18:24.815125 2024 tablet_replica.cc:439] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 9a977e616b4c41bf8dfe52ed84e655d3: TxnStatusTablet state changed. Reason: New leader 84bfbf7596a846f0a69c8c26259f7685. Latest consensus state: current_term: 1 leader_uuid: "84bfbf7596a846f0a69c8c26259f7685" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } } }
I20260811 22:18:24.815433 2024 tablet_replica.cc:442] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 9a977e616b4c41bf8dfe52ed84e655d3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:18:24.820899 2052 mvcc.cc:205] Tried to move back new op lower bound from 7317449542784790528 to 7317449542639431680. Current Snapshot: MvccSnapshot[applied={T|T < 7317449542784790528}]
I20260811 22:18:24.821493 2026 tablet_replica.cc:439] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "84bfbf7596a846f0a69c8c26259f7685" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } } }
I20260811 22:18:24.821884 2026 tablet_replica.cc:442] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:18:24.836751 2055 mvcc.cc:205] Tried to move back new op lower bound from 7317449542784790528 to 7317449542639431680. Current Snapshot: MvccSnapshot[applied={T|T < 7317449542784790528}]
I20260811 22:18:24.842888 2026 tablet_replica.cc:439] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "84bfbf7596a846f0a69c8c26259f7685" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } } }
I20260811 22:18:24.843214 2026 tablet_replica.cc:442] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 84bfbf7596a846f0a69c8c26259f7685: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:18:24.847780 2024 tablet_replica.cc:439] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 9a977e616b4c41bf8dfe52ed84e655d3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "84bfbf7596a846f0a69c8c26259f7685" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } } }
I20260811 22:18:24.848075 2024 tablet_replica.cc:442] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 9a977e616b4c41bf8dfe52ed84e655d3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:18:24.855392 2025 tablet_replica.cc:439] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "84bfbf7596a846f0a69c8c26259f7685" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } } }
I20260811 22:18:24.855687 2025 tablet_replica.cc:442] T 0d2e0e0cafd444dab3fc32c4ced843f7 P 3d16b93155a24aefa9a585715d61d320: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:18:24.857856 2003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.843851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51808 (local address 127.29.229.3:34577)
0811 22:18:24.844046 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:24.844052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:24.844076 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:18:24.844145 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:24.844151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:24.844237 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:24.844344 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:18:24.845141 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.846725 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.854735 (+ 8010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.855489 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.856277 (+ 788us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.856396 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:18:24.856593 (+ 197us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:18:24.857289 (+ 696us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:24.857349 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 22:18:24.857448 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 22:18:24.877358 2003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.871120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:43081 (local address 127.29.229.3:34577)
0811 22:18:24.871862 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:24.871870 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:24.871886 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:18:24.871936 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:24.871940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:24.872050 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:24.872566 (+ 516us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:18:24.872697 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.873862 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.875199 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.875916 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.876039 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.876526 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:18:24.876693 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:18:24.876797 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:24.876954 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 22:18:24.877022 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":633}
I20260811 22:18:24.877530 1889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:24.871104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:34577 (local address 127.29.229.3:43081)
0811 22:18:24.871507 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:24.871530 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:24.871652 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:24.872174 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:24.872181 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:24.872195 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:18:24.872414 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.872428 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.874003 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.874009 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.875050 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:24.875061 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:24.876052 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:24.876057 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:24.876225 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:24.876279 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:18:24.876796 (+ 517us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:18:24.876809 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 22:18:24.876935 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 22:18:24.877088 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":22}
I20260811 22:18:24.888659 1414 raft_consensus.cc:1261] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 FOLLOWER]: Refusing update from remote peer 84bfbf7596a846f0a69c8c26259f7685: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:24.888659 1651 raft_consensus.cc:1261] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320 [term 1 FOLLOWER]: Refusing update from remote peer 84bfbf7596a846f0a69c8c26259f7685: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:24.889331 2061 consensus_queue.cc:1035] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:24.889701 2026 consensus_queue.cc:1035] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:24.977360 1124 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:18:24.977708 1124 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:18:24.982221 1124 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51: Sending DeleteTablet for 3 replicas of tablet 1c276dacabec46fea1e86d6171018d04
I20260811 22:18:24.984046 1360 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c276dacabec46fea1e86d6171018d04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:24 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:58736
I20260811 22:18:24.984226 1611 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c276dacabec46fea1e86d6171018d04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:24 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:57330
I20260811 22:18:24.985603 1861 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c276dacabec46fea1e86d6171018d04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:24 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:51788
I20260811 22:18:24.987289 1124 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:18:24.988411 1124 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:18:24.997485 2076 tablet_replica.cc:330] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3: stopping tablet replica
I20260811 22:18:24.998945 2076 raft_consensus.cc:2229] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:18:25.000910 1862 tablet_service.cc:1467] Processing CreateTablet for tablet 9ef21499836444cda6b594a642cbf0bb (DEFAULT_TABLE table=test-table [id=b5cdb3ee03c14677b47ee8479dd5cae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:25.001362 2080 tablet_replica.cc:330] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685: stopping tablet replica
I20260811 22:18:25.001438 1862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ef21499836444cda6b594a642cbf0bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:25.001892 2080 raft_consensus.cc:2229] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:18:25.000387 1360 tablet_service.cc:1467] Processing CreateTablet for tablet 9ef21499836444cda6b594a642cbf0bb (DEFAULT_TABLE table=test-table [id=b5cdb3ee03c14677b47ee8479dd5cae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:25.002533 1360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ef21499836444cda6b594a642cbf0bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:25.002737 2080 raft_consensus.cc:2258] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:25.003628 2079 tablet_replica.cc:330] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320: stopping tablet replica
I20260811 22:18:25.004082 2079 raft_consensus.cc:2229] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:18:25.004557 2079 raft_consensus.cc:2258] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:25.004590 1608 tablet_service.cc:1467] Processing CreateTablet for tablet 9ef21499836444cda6b594a642cbf0bb (DEFAULT_TABLE table=test-table [id=b5cdb3ee03c14677b47ee8479dd5cae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:25.005157 1608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ef21499836444cda6b594a642cbf0bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:25.006835 2080 ts_tablet_manager.cc:1905] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:25.013993 2014 tablet_bootstrap.cc:492] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3: Bootstrap starting.
I20260811 22:18:25.014974 2080 ts_tablet_manager.cc:1918] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:18:25.015177 2080 log.cc:1196] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/1c276dacabec46fea1e86d6171018d04
I20260811 22:18:25.015676 2080 ts_tablet_manager.cc:1939] T 1c276dacabec46fea1e86d6171018d04 P 84bfbf7596a846f0a69c8c26259f7685: Deleting consensus metadata
I20260811 22:18:25.015895 2014 tablet_bootstrap.cc:654] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:25.017143 2013 tablet_bootstrap.cc:492] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320: Bootstrap starting.
I20260811 22:18:25.017805 1098 catalog_manager.cc:4841] TS 84bfbf7596a846f0a69c8c26259f7685 (127.29.229.3:34577): tablet 1c276dacabec46fea1e86d6171018d04 (table test-table [id=2395c836b28f4e5ca8f56c4262063a16]) successfully deleted
I20260811 22:18:25.019057 2013 tablet_bootstrap.cc:654] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:25.020316 2079 ts_tablet_manager.cc:1905] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:25.022091 2020 tablet_bootstrap.cc:492] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685: Bootstrap starting.
I20260811 22:18:25.023892 2020 tablet_bootstrap.cc:654] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:25.031908 2013 tablet_bootstrap.cc:492] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320: No bootstrap required, opened a new log
I20260811 22:18:25.033450 2020 tablet_bootstrap.cc:492] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685: No bootstrap required, opened a new log
I20260811 22:18:25.033653 2020 ts_tablet_manager.cc:1397] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260811 22:18:25.032814 2013 ts_tablet_manager.cc:1397] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.005s
I20260811 22:18:25.034621 2079 ts_tablet_manager.cc:1918] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:18:25.034571 2020 raft_consensus.cc:348] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:25.034821 2079 log.cc:1196] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/1c276dacabec46fea1e86d6171018d04
I20260811 22:18:25.034919 2020 raft_consensus.cc:374] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:25.035028 2020 raft_consensus.cc:729] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84bfbf7596a846f0a69c8c26259f7685, State: Initialized, Role: FOLLOWER
I20260811 22:18:25.035449 2079 ts_tablet_manager.cc:1939] T 1c276dacabec46fea1e86d6171018d04 P 3d16b93155a24aefa9a585715d61d320: Deleting consensus metadata
I20260811 22:18:25.035370 2020 consensus_queue.cc:260] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:25.035786 2013 raft_consensus.cc:348] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:25.036111 2013 raft_consensus.cc:374] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:25.036273 2013 raft_consensus.cc:729] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d16b93155a24aefa9a585715d61d320, State: Initialized, Role: FOLLOWER
I20260811 22:18:25.036693 2013 consensus_queue.cc:260] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:25.037863 2020 ts_tablet_manager.cc:1428] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260811 22:18:25.039254 1100 catalog_manager.cc:4841] TS 3d16b93155a24aefa9a585715d61d320 (127.29.229.2:40199): tablet 1c276dacabec46fea1e86d6171018d04 (table test-table [id=2395c836b28f4e5ca8f56c4262063a16]) successfully deleted
I20260811 22:18:25.040289 2013 ts_tablet_manager.cc:1428] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260811 22:18:25.041780 2076 raft_consensus.cc:2258] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:25.043483 2014 tablet_bootstrap.cc:492] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3: No bootstrap required, opened a new log
I20260811 22:18:25.043708 2014 ts_tablet_manager.cc:1397] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3: Time spent bootstrapping tablet: real 0.030s user 0.012s sys 0.003s
I20260811 22:18:25.044584 2014 raft_consensus.cc:348] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:25.044839 2014 raft_consensus.cc:374] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:25.044952 2014 raft_consensus.cc:729] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a977e616b4c41bf8dfe52ed84e655d3, State: Initialized, Role: FOLLOWER
I20260811 22:18:25.045226 2014 consensus_queue.cc:260] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:25.045681 2076 ts_tablet_manager.cc:1905] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:25.045819 2014 ts_tablet_manager.cc:1428] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:18:25.053747 2076 ts_tablet_manager.cc:1918] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:18:25.053961 2076 log.cc:1196] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/wal/wals/1c276dacabec46fea1e86d6171018d04
I20260811 22:18:25.054530 2076 ts_tablet_manager.cc:1939] T 1c276dacabec46fea1e86d6171018d04 P 9a977e616b4c41bf8dfe52ed84e655d3: Deleting consensus metadata
I20260811 22:18:25.056759 1098 catalog_manager.cc:4841] TS 9a977e616b4c41bf8dfe52ed84e655d3 (127.29.229.1:35515): tablet 1c276dacabec46fea1e86d6171018d04 (table test-table [id=2395c836b28f4e5ca8f56c4262063a16]) successfully deleted
I20260811 22:18:25.084862 2024 raft_consensus.cc:482] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:18:25.085126 2024 raft_consensus.cc:504] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:25.086714 2024 leader_election.cc:288] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d16b93155a24aefa9a585715d61d320 (127.29.229.2:40199), 84bfbf7596a846f0a69c8c26259f7685 (127.29.229.3:34577)
I20260811 22:18:25.095520 2003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:25.087707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:51987 (local address 127.29.229.3:34577)
0811 22:18:25.087910 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:25.087916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:25.088735 (+ 819us) server_negotiation.cc:407] Connection header received
0811 22:18:25.088891 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:25.088896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:25.088993 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:25.089097 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:25.089884 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:25.091022 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:25.092978 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:25.094335 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:25.094525 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:25.094605 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:25.095018 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 22:18:25.095181 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:18:25.095711 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:25.087160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:37841 (local address 127.29.229.2:40199)
0811 22:18:25.087408 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:25.087431 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:25.088078 (+ 647us) server_negotiation.cc:407] Connection header received
0811 22:18:25.088435 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:25.088443 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:25.088555 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:25.088619 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:25.089482 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:25.090680 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:25.093318 (+ 2638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:25.094591 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:25.094770 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:25.094852 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:25.095243 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 22:18:25.095395 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:18:25.095911 2085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:25.087148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:34577 (local address 127.29.229.1:51987)
0811 22:18:25.088603 (+ 1455us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:25.088621 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:25.088756 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:25.089148 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:25.089154 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:25.089180 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:25.089681 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:25.089694 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:25.091161 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:25.091167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:25.092821 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:25.092835 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:25.094599 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:25.094606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:25.094813 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:25.094850 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 22:18:25.094971 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 22:18:25.095130 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1286,"mutex_wait_us":3,"thread_start_us":757,"threads_started":1}
I20260811 22:18:25.096537 2086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:25.086945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:40199 (local address 127.29.229.1:37841)
0811 22:18:25.087935 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:25.087961 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:25.088223 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:25.088853 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:25.088858 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:25.088887 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:25.089300 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:25.089316 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:25.090852 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:25.090857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:25.093141 (+ 2284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:25.093158 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:25.094836 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:25.094842 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:25.095048 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:25.095083 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 22:18:25.095950 (+ 867us) client_negotiation.cc:240] Negotiation successful
0811 22:18:25.096112 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"mutex_wait_us":5,"thread_start_us":1336,"threads_started":1}
I20260811 22:18:25.097456 1651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ef21499836444cda6b594a642cbf0bb" candidate_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d16b93155a24aefa9a585715d61d320" is_pre_election: true
I20260811 22:18:25.097347 1897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ef21499836444cda6b594a642cbf0bb" candidate_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84bfbf7596a846f0a69c8c26259f7685" is_pre_election: true
I20260811 22:18:25.097744 1651 raft_consensus.cc:2454] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a977e616b4c41bf8dfe52ed84e655d3 in term 0.
I20260811 22:18:25.097946 1897 raft_consensus.cc:2454] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a977e616b4c41bf8dfe52ed84e655d3 in term 0.
I20260811 22:18:25.098373 1280 leader_election.cc:302] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d16b93155a24aefa9a585715d61d320, 9a977e616b4c41bf8dfe52ed84e655d3; no voters:
I20260811 22:18:25.099025 2024 raft_consensus.cc:2781] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:18:25.099136 2024 raft_consensus.cc:482] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:18:25.099210 2024 raft_consensus.cc:3037] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:25.101760 2024 raft_consensus.cc:504] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:25.103125 1651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ef21499836444cda6b594a642cbf0bb" candidate_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d16b93155a24aefa9a585715d61d320"
I20260811 22:18:25.103135 1897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ef21499836444cda6b594a642cbf0bb" candidate_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84bfbf7596a846f0a69c8c26259f7685"
I20260811 22:18:25.103374 1651 raft_consensus.cc:3037] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:25.103396 1897 raft_consensus.cc:3037] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:25.104667 2024 leader_election.cc:288] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [CANDIDATE]: Term 1 election: Requested vote from peers 3d16b93155a24aefa9a585715d61d320 (127.29.229.2:40199), 84bfbf7596a846f0a69c8c26259f7685 (127.29.229.3:34577)
I20260811 22:18:25.106592 1651 raft_consensus.cc:2454] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a977e616b4c41bf8dfe52ed84e655d3 in term 1.
I20260811 22:18:25.107197 1280 leader_election.cc:302] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d16b93155a24aefa9a585715d61d320, 9a977e616b4c41bf8dfe52ed84e655d3; no voters:
I20260811 22:18:25.107295 1897 raft_consensus.cc:2454] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a977e616b4c41bf8dfe52ed84e655d3 in term 1.
I20260811 22:18:25.107596 2024 raft_consensus.cc:2781] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:25.108256 2024 raft_consensus.cc:686] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 LEADER]: Becoming Leader. State: Replica: 9a977e616b4c41bf8dfe52ed84e655d3, State: Running, Role: LEADER
I20260811 22:18:25.108944 2024 consensus_queue.cc:237] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:25.112963 1124 catalog_manager.cc:5495] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a977e616b4c41bf8dfe52ed84e655d3 (127.29.229.1). New cstate: current_term: 1 leader_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } health_report { overall_health: UNKNOWN } } }
I20260811 22:18:25.152904 2094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:25.145936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58748 (local address 127.29.229.1:35515)
0811 22:18:25.146544 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:25.146548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:25.146564 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:18:25.146644 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:25.146648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:25.146721 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:25.146865 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:18:25.147356 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:25.148111 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:25.150051 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:25.150495 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:25.150606 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:25.151226 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:18:25.151806 (+ 580us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:18:25.151926 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:25.152359 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 22:18:25.152439 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":346,"threads_started":1}
I20260811 22:18:25.179452 2003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:25.164859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:45649 (local address 127.29.229.3:34577)
0811 22:18:25.165128 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:25.165135 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:25.172021 (+ 6886us) server_negotiation.cc:407] Connection header received
0811 22:18:25.172437 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:25.172445 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:25.172570 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:25.172665 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:18:25.173431 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:25.174648 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:25.177003 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:25.177714 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:25.177855 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:25.178496 (+ 641us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:18:25.178630 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:18:25.178724 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:25.179037 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 22:18:25.179130 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:18:25.179855 2098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:25.164827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:34577 (local address 127.29.229.1:45649)
0811 22:18:25.171847 (+ 7020us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:25.171876 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:25.172224 (+ 348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:25.172716 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:25.172722 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:25.172739 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:18:25.173227 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:25.173245 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:25.174833 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:25.174839 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:25.176829 (+ 1990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:25.176851 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:25.177984 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:25.177994 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:25.178197 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:25.178286 (+ 89us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:18:25.178776 (+ 490us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:18:25.178792 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 22:18:25.178884 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 22:18:25.179117 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6723,"thread_start_us":6309,"threads_started":1}
I20260811 22:18:25.193739 1651 raft_consensus.cc:1261] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320 [term 1 FOLLOWER]: Refusing update from remote peer 9a977e616b4c41bf8dfe52ed84e655d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:25.193979 1897 raft_consensus.cc:1261] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685 [term 1 FOLLOWER]: Refusing update from remote peer 9a977e616b4c41bf8dfe52ed84e655d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:25.194816 2087 consensus_queue.cc:1035] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:25.195341 2024 consensus_queue.cc:1035] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:25.234431 2094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:25.227842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:44367 (local address 127.29.229.1:35515)
0811 22:18:25.228494 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:25.228501 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:25.228521 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:18:25.228585 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:25.228590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:25.228676 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:25.228780 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:18:25.229392 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:25.230252 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:25.232223 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:25.232883 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:25.233046 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:25.233596 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:18:25.233736 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:18:25.233836 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:25.234032 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 22:18:25.234120 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494}
I20260811 22:18:25.234880 1889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:25.227705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:35515 (local address 127.29.229.3:44367)
0811 22:18:25.228066 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:25.228090 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:25.228222 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:25.228968 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:25.228973 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:25.228987 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:18:25.229227 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:25.229240 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:25.230413 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:25.230420 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:25.232024 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:25.232064 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:25.233192 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:25.233201 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:25.233364 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:25.233408 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:18:25.233869 (+ 461us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:18:25.233884 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 22:18:25.233988 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 22:18:25.234166 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
W20260811 22:18:25.236832 2055 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 22:18:26.845542 1122 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:18:26.845824 1122 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:18:26.850410 1122 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51: Sending DeleteTablet for 3 replicas of tablet 9ef21499836444cda6b594a642cbf0bb
I20260811 22:18:26.851428 1862 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ef21499836444cda6b594a642cbf0bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:26 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:51788
I20260811 22:18:26.851413 1360 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ef21499836444cda6b594a642cbf0bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:26 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:58736
I20260811 22:18:26.851697 1611 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ef21499836444cda6b594a642cbf0bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:26 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:57330
I20260811 22:18:26.864575 2155 tablet_replica.cc:330] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685: stopping tablet replica
I20260811 22:18:26.865674 2155 raft_consensus.cc:2229] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:18:26.866403 2155 raft_consensus.cc:2258] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:26.867691 2152 tablet_replica.cc:330] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3: stopping tablet replica
I20260811 22:18:26.868110 2152 raft_consensus.cc:2229] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:18:26.868518 2161 tablet_replica.cc:330] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320: stopping tablet replica
I20260811 22:18:26.868780 2161 raft_consensus.cc:2229] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:18:26.869012 2152 raft_consensus.cc:2258] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:26.869072 2161 raft_consensus.cc:2258] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:26.870186 2152 ts_tablet_manager.cc:1905] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:26.870179 2155 ts_tablet_manager.cc:1905] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:26.870729 2161 ts_tablet_manager.cc:1905] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:26.878091 2152 ts_tablet_manager.cc:1918] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:18:26.878091 2155 ts_tablet_manager.cc:1918] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:18:26.878269 2152 log.cc:1196] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/wal/wals/9ef21499836444cda6b594a642cbf0bb
I20260811 22:18:26.878269 2155 log.cc:1196] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/9ef21499836444cda6b594a642cbf0bb
I20260811 22:18:26.878758 2155 ts_tablet_manager.cc:1939] T 9ef21499836444cda6b594a642cbf0bb P 84bfbf7596a846f0a69c8c26259f7685: Deleting consensus metadata
I20260811 22:18:26.878758 2152 ts_tablet_manager.cc:1939] T 9ef21499836444cda6b594a642cbf0bb P 9a977e616b4c41bf8dfe52ed84e655d3: Deleting consensus metadata
I20260811 22:18:26.878736 2161 ts_tablet_manager.cc:1918] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:18:26.878890 2161 log.cc:1196] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/9ef21499836444cda6b594a642cbf0bb
I20260811 22:18:26.879395 2161 ts_tablet_manager.cc:1939] T 9ef21499836444cda6b594a642cbf0bb P 3d16b93155a24aefa9a585715d61d320: Deleting consensus metadata
I20260811 22:18:26.880090 1098 catalog_manager.cc:4841] TS 84bfbf7596a846f0a69c8c26259f7685 (127.29.229.3:34577): tablet 9ef21499836444cda6b594a642cbf0bb (table test-table [id=b5cdb3ee03c14677b47ee8479dd5cae3]) successfully deleted
I20260811 22:18:26.880370 1098 catalog_manager.cc:4841] TS 9a977e616b4c41bf8dfe52ed84e655d3 (127.29.229.1:35515): tablet 9ef21499836444cda6b594a642cbf0bb (table test-table [id=b5cdb3ee03c14677b47ee8479dd5cae3]) successfully deleted
I20260811 22:18:26.880903 1100 catalog_manager.cc:4841] TS 3d16b93155a24aefa9a585715d61d320 (127.29.229.2:40199): tablet 9ef21499836444cda6b594a642cbf0bb (table test-table [id=b5cdb3ee03c14677b47ee8479dd5cae3]) successfully deleted
Aug 11 22:18:26 dist-test-slave-jq4t krb5kdc[1052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486706, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:18:26 dist-test-slave-jq4t krb5kdc[1052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486706, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:26.928937 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:26.911223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43608 (local address 127.29.229.62:43861)
0811 22:18:26.911939 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:26.911944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:26.911969 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:18:26.912131 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:26.912135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:26.912201 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:26.912301 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:26.913940 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:26.914754 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:26.916105 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:26.916623 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:26.916731 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:26.921232 (+ 4501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:26.921238 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:26.921241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:26.921279 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:26.925230 (+ 3951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:26.926305 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:26.926312 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:26.926317 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:26.926499 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:26.926958 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:26.926962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:26.926965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:26.927450 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:26.927727 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:26.928229 (+ 502us) server_negotiation.cc:299] Negotiation successful
0811 22:18:26.928472 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":365,"threads_started":1}
I20260811 22:18:26.933640 1122 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43608:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:18:26.934307 1122 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:18:26.941983 1360 tablet_service.cc:1467] Processing CreateTablet for tablet 0c75fd48cd6447b29040c3150ca10a78 (DEFAULT_TABLE table=test-table [id=1a789745691945999c311d9001c655e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:26.942166 1611 tablet_service.cc:1467] Processing CreateTablet for tablet 0c75fd48cd6447b29040c3150ca10a78 (DEFAULT_TABLE table=test-table [id=1a789745691945999c311d9001c655e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:26.942229 1862 tablet_service.cc:1467] Processing CreateTablet for tablet 0c75fd48cd6447b29040c3150ca10a78 (DEFAULT_TABLE table=test-table [id=1a789745691945999c311d9001c655e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:26.942515 1360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c75fd48cd6447b29040c3150ca10a78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:26.942595 1611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c75fd48cd6447b29040c3150ca10a78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:26.942762 1862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c75fd48cd6447b29040c3150ca10a78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:26.948407 2175 tablet_bootstrap.cc:492] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685: Bootstrap starting.
I20260811 22:18:26.949929 2175 tablet_bootstrap.cc:654] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:26.950639 2180 tablet_bootstrap.cc:492] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3: Bootstrap starting.
I20260811 22:18:26.951261 2179 tablet_bootstrap.cc:492] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320: Bootstrap starting.
I20260811 22:18:26.953192 2180 tablet_bootstrap.cc:654] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:26.953203 2179 tablet_bootstrap.cc:654] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:26.953781 2175 tablet_bootstrap.cc:492] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685: No bootstrap required, opened a new log
I20260811 22:18:26.954005 2175 ts_tablet_manager.cc:1397] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260811 22:18:26.955173 2175 raft_consensus.cc:348] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:26.955468 2175 raft_consensus.cc:374] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:26.955592 2175 raft_consensus.cc:729] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84bfbf7596a846f0a69c8c26259f7685, State: Initialized, Role: FOLLOWER
I20260811 22:18:26.955879 2175 consensus_queue.cc:260] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:26.956562 2175 ts_tablet_manager.cc:1428] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 22:18:26.958940 2180 tablet_bootstrap.cc:492] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3: No bootstrap required, opened a new log
I20260811 22:18:26.959110 2180 ts_tablet_manager.cc:1397] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260811 22:18:26.959136 2179 tablet_bootstrap.cc:492] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320: No bootstrap required, opened a new log
I20260811 22:18:26.959311 2179 ts_tablet_manager.cc:1397] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 22:18:26.959996 2180 raft_consensus.cc:348] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:26.960258 2180 raft_consensus.cc:374] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:26.960373 2180 raft_consensus.cc:729] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a977e616b4c41bf8dfe52ed84e655d3, State: Initialized, Role: FOLLOWER
I20260811 22:18:26.960188 2179 raft_consensus.cc:348] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:26.960767 2179 raft_consensus.cc:374] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:26.960685 2180 consensus_queue.cc:260] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:26.961300 2180 ts_tablet_manager.cc:1428] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:18:26.960887 2179 raft_consensus.cc:729] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d16b93155a24aefa9a585715d61d320, State: Initialized, Role: FOLLOWER
I20260811 22:18:26.961736 2179 consensus_queue.cc:260] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:26.962654 2179 ts_tablet_manager.cc:1428] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260811 22:18:27.116844 2125 raft_consensus.cc:482] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:18:27.117097 2125 raft_consensus.cc:504] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:27.117750 2125 leader_election.cc:288] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d16b93155a24aefa9a585715d61d320 (127.29.229.2:40199), 84bfbf7596a846f0a69c8c26259f7685 (127.29.229.3:34577)
I20260811 22:18:27.118587 1894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c75fd48cd6447b29040c3150ca10a78" candidate_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84bfbf7596a846f0a69c8c26259f7685" is_pre_election: true
I20260811 22:18:27.118595 1649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c75fd48cd6447b29040c3150ca10a78" candidate_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d16b93155a24aefa9a585715d61d320" is_pre_election: true
I20260811 22:18:27.118871 1649 raft_consensus.cc:2454] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a977e616b4c41bf8dfe52ed84e655d3 in term 0.
I20260811 22:18:27.118891 1894 raft_consensus.cc:2454] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a977e616b4c41bf8dfe52ed84e655d3 in term 0.
I20260811 22:18:27.119479 1275 leader_election.cc:302] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84bfbf7596a846f0a69c8c26259f7685, 9a977e616b4c41bf8dfe52ed84e655d3; no voters:
I20260811 22:18:27.119791 2125 raft_consensus.cc:2781] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:18:27.119908 2125 raft_consensus.cc:482] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:18:27.120023 2125 raft_consensus.cc:3037] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:27.122980 2125 raft_consensus.cc:504] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:27.123683 2125 leader_election.cc:288] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [CANDIDATE]: Term 1 election: Requested vote from peers 3d16b93155a24aefa9a585715d61d320 (127.29.229.2:40199), 84bfbf7596a846f0a69c8c26259f7685 (127.29.229.3:34577)
I20260811 22:18:27.124255 1649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c75fd48cd6447b29040c3150ca10a78" candidate_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d16b93155a24aefa9a585715d61d320"
I20260811 22:18:27.124265 1894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c75fd48cd6447b29040c3150ca10a78" candidate_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84bfbf7596a846f0a69c8c26259f7685"
I20260811 22:18:27.124505 1649 raft_consensus.cc:3037] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:27.124576 1894 raft_consensus.cc:3037] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:27.126880 1649 raft_consensus.cc:2454] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a977e616b4c41bf8dfe52ed84e655d3 in term 1.
I20260811 22:18:27.127331 1894 raft_consensus.cc:2454] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a977e616b4c41bf8dfe52ed84e655d3 in term 1.
I20260811 22:18:27.127394 1280 leader_election.cc:302] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d16b93155a24aefa9a585715d61d320, 9a977e616b4c41bf8dfe52ed84e655d3; no voters:
I20260811 22:18:27.127689 2125 raft_consensus.cc:2781] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:27.127892 2125 raft_consensus.cc:686] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 LEADER]: Becoming Leader. State: Replica: 9a977e616b4c41bf8dfe52ed84e655d3, State: Running, Role: LEADER
I20260811 22:18:27.128206 2125 consensus_queue.cc:237] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:27.131817 1122 catalog_manager.cc:5495] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a977e616b4c41bf8dfe52ed84e655d3 (127.29.229.1). New cstate: current_term: 1 leader_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } health_report { overall_health: UNKNOWN } } }
I20260811 22:18:27.160532 2194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:27.154731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43958 (local address 127.29.229.1:35515)
0811 22:18:27.155381 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:27.155386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:27.155406 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:18:27.155460 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:27.155463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:27.155526 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:27.155615 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:18:27.156022 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.157168 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.158559 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.159043 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.159142 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:27.159609 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:18:27.159720 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:18:27.159810 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:27.160061 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 22:18:27.160169 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":416,"threads_started":1}
I20260811 22:18:27.172082 1894 raft_consensus.cc:1261] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685 [term 1 FOLLOWER]: Refusing update from remote peer 9a977e616b4c41bf8dfe52ed84e655d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:27.172082 1649 raft_consensus.cc:1261] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320 [term 1 FOLLOWER]: Refusing update from remote peer 9a977e616b4c41bf8dfe52ed84e655d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:27.172739 2137 consensus_queue.cc:1035] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:27.173134 2125 consensus_queue.cc:1035] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:27.228623 1124 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:18:27.228957 1124 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:18:27.233069 1124 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51: Sending DeleteTablet for 3 replicas of tablet 0c75fd48cd6447b29040c3150ca10a78
I20260811 22:18:27.234009 1360 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c75fd48cd6447b29040c3150ca10a78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:27 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:58736
I20260811 22:18:27.234117 1611 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c75fd48cd6447b29040c3150ca10a78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:27 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:57330
I20260811 22:18:27.234092 1862 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c75fd48cd6447b29040c3150ca10a78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:27 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:51788
I20260811 22:18:27.234360 2152 tablet_replica.cc:330] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3: stopping tablet replica
I20260811 22:18:27.234417 2155 tablet_replica.cc:330] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685: stopping tablet replica
I20260811 22:18:27.234413 2161 tablet_replica.cc:330] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320: stopping tablet replica
I20260811 22:18:27.234601 2152 raft_consensus.cc:2229] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:18:27.234633 2155 raft_consensus.cc:2229] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:18:27.234633 2161 raft_consensus.cc:2229] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:18:27.234838 2155 raft_consensus.cc:2258] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:27.234848 2161 raft_consensus.cc:2258] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:27.234974 2152 raft_consensus.cc:2258] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:27.235708 1124 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43608:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 22:18:27.236112 2155 ts_tablet_manager.cc:1905] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:27.236243 2152 ts_tablet_manager.cc:1905] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:27.236384 2161 ts_tablet_manager.cc:1905] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 22:18:27.236811 1124 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:18:27.243029 2152 ts_tablet_manager.cc:1918] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:18:27.243126 2155 ts_tablet_manager.cc:1918] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:18:27.243201 2152 log.cc:1196] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/wal/wals/0c75fd48cd6447b29040c3150ca10a78
I20260811 22:18:27.243319 2155 log.cc:1196] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/0c75fd48cd6447b29040c3150ca10a78
I20260811 22:18:27.243667 2152 ts_tablet_manager.cc:1939] T 0c75fd48cd6447b29040c3150ca10a78 P 9a977e616b4c41bf8dfe52ed84e655d3: Deleting consensus metadata
I20260811 22:18:27.243799 2155 ts_tablet_manager.cc:1939] T 0c75fd48cd6447b29040c3150ca10a78 P 84bfbf7596a846f0a69c8c26259f7685: Deleting consensus metadata
I20260811 22:18:27.244434 2161 ts_tablet_manager.cc:1918] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:18:27.244968 2161 log.cc:1196] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/0c75fd48cd6447b29040c3150ca10a78
I20260811 22:18:27.244951 1098 catalog_manager.cc:4841] TS 84bfbf7596a846f0a69c8c26259f7685 (127.29.229.3:34577): tablet 0c75fd48cd6447b29040c3150ca10a78 (table test-table [id=1a789745691945999c311d9001c655e5]) successfully deleted
I20260811 22:18:27.245513 2161 ts_tablet_manager.cc:1939] T 0c75fd48cd6447b29040c3150ca10a78 P 3d16b93155a24aefa9a585715d61d320: Deleting consensus metadata
I20260811 22:18:27.245544 1611 tablet_service.cc:1467] Processing CreateTablet for tablet 8d5c24a61f314d57a7591b50209c21ce (DEFAULT_TABLE table=test-table [id=7c0b19e2736d43229ee9e005661c587f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:27.245851 1862 tablet_service.cc:1467] Processing CreateTablet for tablet 8d5c24a61f314d57a7591b50209c21ce (DEFAULT_TABLE table=test-table [id=7c0b19e2736d43229ee9e005661c587f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:27.246031 1611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d5c24a61f314d57a7591b50209c21ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:27.246243 1862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d5c24a61f314d57a7591b50209c21ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:27.246508 1360 tablet_service.cc:1467] Processing CreateTablet for tablet 8d5c24a61f314d57a7591b50209c21ce (DEFAULT_TABLE table=test-table [id=7c0b19e2736d43229ee9e005661c587f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:27.246964 1360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d5c24a61f314d57a7591b50209c21ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:27.250900 1100 catalog_manager.cc:4841] TS 3d16b93155a24aefa9a585715d61d320 (127.29.229.2:40199): tablet 0c75fd48cd6447b29040c3150ca10a78 (table test-table [id=1a789745691945999c311d9001c655e5]) successfully deleted
I20260811 22:18:27.251503 2180 tablet_bootstrap.cc:492] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3: Bootstrap starting.
I20260811 22:18:27.252628 1098 catalog_manager.cc:4841] TS 9a977e616b4c41bf8dfe52ed84e655d3 (127.29.229.1:35515): tablet 0c75fd48cd6447b29040c3150ca10a78 (table test-table [id=1a789745691945999c311d9001c655e5]) successfully deleted
I20260811 22:18:27.253234 2179 tablet_bootstrap.cc:492] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320: Bootstrap starting.
I20260811 22:18:27.253338 2180 tablet_bootstrap.cc:654] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:27.253579 2175 tablet_bootstrap.cc:492] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685: Bootstrap starting.
I20260811 22:18:27.254967 2179 tablet_bootstrap.cc:654] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:27.255347 2175 tablet_bootstrap.cc:654] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:27.258148 2180 tablet_bootstrap.cc:492] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3: No bootstrap required, opened a new log
I20260811 22:18:27.258373 2175 tablet_bootstrap.cc:492] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685: No bootstrap required, opened a new log
I20260811 22:18:27.258342 2180 ts_tablet_manager.cc:1397] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260811 22:18:27.258589 2175 ts_tablet_manager.cc:1397] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260811 22:18:27.258806 2179 tablet_bootstrap.cc:492] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320: No bootstrap required, opened a new log
I20260811 22:18:27.258972 2179 ts_tablet_manager.cc:1397] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260811 22:18:27.259193 2180 raft_consensus.cc:348] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:27.259333 2175 raft_consensus.cc:348] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:27.259486 2180 raft_consensus.cc:374] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:27.259568 2180 raft_consensus.cc:729] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a977e616b4c41bf8dfe52ed84e655d3, State: Initialized, Role: FOLLOWER
I20260811 22:18:27.259596 2175 raft_consensus.cc:374] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:27.259678 2175 raft_consensus.cc:729] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84bfbf7596a846f0a69c8c26259f7685, State: Initialized, Role: FOLLOWER
I20260811 22:18:27.259773 2179 raft_consensus.cc:348] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:27.259846 2180 consensus_queue.cc:260] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:27.260087 2179 raft_consensus.cc:374] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:27.259990 2175 consensus_queue.cc:260] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:27.260169 2179 raft_consensus.cc:729] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d16b93155a24aefa9a585715d61d320, State: Initialized, Role: FOLLOWER
I20260811 22:18:27.260430 2179 consensus_queue.cc:260] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:27.261025 2179 ts_tablet_manager.cc:1428] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:18:27.261500 2175 ts_tablet_manager.cc:1428] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 22:18:27.260434 2180 ts_tablet_manager.cc:1428] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260811 22:18:27.470403 2185 raft_consensus.cc:482] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:18:27.470671 2185 raft_consensus.cc:504] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:27.472282 2185 leader_election.cc:288] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a977e616b4c41bf8dfe52ed84e655d3 (127.29.229.1:35515), 84bfbf7596a846f0a69c8c26259f7685 (127.29.229.3:34577)
I20260811 22:18:27.480329 2194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:27.472693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:47791 (local address 127.29.229.1:35515)
0811 22:18:27.472943 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:27.472953 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:27.473523 (+ 570us) server_negotiation.cc:407] Connection header received
0811 22:18:27.473742 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:27.473748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:27.473834 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:27.473935 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:27.474726 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.475766 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.478039 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.479292 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.479465 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:27.479544 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:27.479887 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 22:18:27.480023 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:18:27.480501 2207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:27.472651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:35515 (local address 127.29.229.2:47791)
0811 22:18:27.473382 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:27.473405 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:27.473578 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:27.474129 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:27.474134 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:27.474162 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:27.474564 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:27.474581 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.475905 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.475910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:27.477854 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:27.477870 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.479510 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.479517 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:27.479694 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:27.479732 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 22:18:27.479843 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 22:18:27.480021 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":27,"thread_start_us":400,"threads_started":1}
I20260811 22:18:27.481217 2210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:27.473462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:34577 (local address 127.29.229.2:50441)
0811 22:18:27.474036 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:27.474052 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:27.474155 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:27.474984 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:27.474989 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:27.475017 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:27.475393 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:27.475404 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.476819 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.476825 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:27.478625 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:27.478637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.480081 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.480087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:27.480274 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:27.480306 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:18:27.480384 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:18:27.480861 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":1,"thread_start_us":579,"threads_started":1}
I20260811 22:18:27.481303 1405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d5c24a61f314d57a7591b50209c21ce" candidate_uuid: "3d16b93155a24aefa9a585715d61d320" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" is_pre_election: true
I20260811 22:18:27.481598 1405 raft_consensus.cc:2454] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d16b93155a24aefa9a585715d61d320 in term 0.
I20260811 22:18:27.482184 1539 leader_election.cc:302] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d16b93155a24aefa9a585715d61d320, 9a977e616b4c41bf8dfe52ed84e655d3; no voters:
I20260811 22:18:27.482340 2212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:27.473925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:50441 (local address 127.29.229.3:34577)
0811 22:18:27.474670 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:27.474677 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:27.474701 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:18:27.474769 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:27.474774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:27.474844 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:27.474941 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:27.475571 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.476676 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.478797 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.479865 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.481657 (+ 1792us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:27.481737 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:27.481793 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 22:18:27.481971 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":476,"threads_started":1}
I20260811 22:18:27.482614 2185 raft_consensus.cc:2781] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:18:27.482752 2185 raft_consensus.cc:482] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:18:27.482977 2185 raft_consensus.cc:3037] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:27.483454 1894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d5c24a61f314d57a7591b50209c21ce" candidate_uuid: "3d16b93155a24aefa9a585715d61d320" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84bfbf7596a846f0a69c8c26259f7685" is_pre_election: true
I20260811 22:18:27.483981 1894 raft_consensus.cc:2454] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d16b93155a24aefa9a585715d61d320 in term 0.
I20260811 22:18:27.485697 2185 raft_consensus.cc:504] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:27.486367 2185 leader_election.cc:288] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [CANDIDATE]: Term 1 election: Requested vote from peers 9a977e616b4c41bf8dfe52ed84e655d3 (127.29.229.1:35515), 84bfbf7596a846f0a69c8c26259f7685 (127.29.229.3:34577)
I20260811 22:18:27.486819 1405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d5c24a61f314d57a7591b50209c21ce" candidate_uuid: "3d16b93155a24aefa9a585715d61d320" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a977e616b4c41bf8dfe52ed84e655d3"
I20260811 22:18:27.487061 1405 raft_consensus.cc:3037] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:27.486984 1894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d5c24a61f314d57a7591b50209c21ce" candidate_uuid: "3d16b93155a24aefa9a585715d61d320" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84bfbf7596a846f0a69c8c26259f7685"
I20260811 22:18:27.487191 1894 raft_consensus.cc:3037] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:27.489310 1405 raft_consensus.cc:2454] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d16b93155a24aefa9a585715d61d320 in term 1.
I20260811 22:18:27.489779 1539 leader_election.cc:302] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d16b93155a24aefa9a585715d61d320, 9a977e616b4c41bf8dfe52ed84e655d3; no voters:
I20260811 22:18:27.490091 2185 raft_consensus.cc:2781] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:27.490466 1894 raft_consensus.cc:2454] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d16b93155a24aefa9a585715d61d320 in term 1.
I20260811 22:18:27.490988 2185 raft_consensus.cc:686] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [term 1 LEADER]: Becoming Leader. State: Replica: 3d16b93155a24aefa9a585715d61d320, State: Running, Role: LEADER
I20260811 22:18:27.491652 2185 consensus_queue.cc:237] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } }
I20260811 22:18:27.495071 1123 catalog_manager.cc:5495] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d16b93155a24aefa9a585715d61d320 (127.29.229.2). New cstate: current_term: 1 leader_uuid: "3d16b93155a24aefa9a585715d61d320" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d16b93155a24aefa9a585715d61d320" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 } health_report { overall_health: UNKNOWN } } }
I20260811 22:18:27.574957 2217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:27.568573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37992 (local address 127.29.229.2:40199)
0811 22:18:27.569250 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:27.569255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:27.569283 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:18:27.569369 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:27.569373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:27.569464 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:27.569649 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:18:27.570087 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.570944 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.572546 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.572990 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.573143 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:27.573617 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:18:27.574249 (+ 632us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:18:27.574405 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:27.574506 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 22:18:27.574599 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":341,"threads_started":1}
I20260811 22:18:27.591346 2212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:27.585647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:34009 (local address 127.29.229.3:34577)
0811 22:18:27.585897 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:27.585905 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:27.586309 (+ 404us) server_negotiation.cc:407] Connection header received
0811 22:18:27.586541 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:27.586546 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:27.586614 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:27.586710 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:18:27.587244 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.587952 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.589489 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.589931 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.590074 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:27.590574 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:18:27.590691 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:18:27.590790 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:27.590957 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 22:18:27.591040 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:18:27.591470 2222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:27.585401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:34577 (local address 127.29.229.2:34009)
0811 22:18:27.586201 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:27.586217 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:27.586394 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:27.586741 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:27.586746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:27.586758 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:18:27.587095 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:27.587103 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.588093 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.588098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:27.589335 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:27.589349 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.590130 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.590137 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:27.590320 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:27.590390 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:18:27.590809 (+ 419us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:18:27.590830 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:18:27.590931 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 22:18:27.591096 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":467,"threads_started":1}
I20260811 22:18:27.600131 1894 raft_consensus.cc:1261] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685 [term 1 FOLLOWER]: Refusing update from remote peer 3d16b93155a24aefa9a585715d61d320: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:27.600198 1405 raft_consensus.cc:1261] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 FOLLOWER]: Refusing update from remote peer 3d16b93155a24aefa9a585715d61d320: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:27.600958 2214 consensus_queue.cc:1035] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a977e616b4c41bf8dfe52ed84e655d3" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 35515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:27.601332 2185 consensus_queue.cc:1035] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84bfbf7596a846f0a69c8c26259f7685" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 34577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:27.605525 2113 mvcc.cc:205] Tried to move back new op lower bound from 7317449554324619264 to 7317449553890123776. Current Snapshot: MvccSnapshot[applied={T|T < 7317449554324619264}]
I20260811 22:18:27.632988 2217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:27.626291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:49187 (local address 127.29.229.2:40199)
0811 22:18:27.626477 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:27.626483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:27.627112 (+ 629us) server_negotiation.cc:407] Connection header received
0811 22:18:27.627237 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:27.627243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:27.627320 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:27.627398 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:18:27.627918 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.628883 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.630773 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.631330 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.631468 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:27.632108 (+ 640us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:18:27.632247 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:18:27.632383 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:27.632581 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 22:18:27.632665 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:18:27.633126 2232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:27.626266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:40199 (local address 127.29.229.3:49187)
0811 22:18:27.627007 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:27.627023 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:27.627136 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:27.627424 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:27.627427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:27.627436 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:18:27.627754 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:27.627764 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.629055 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.629059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:27.630599 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:27.630613 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:27.631555 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:27.631564 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:27.631756 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:27.631841 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:18:27.632385 (+ 544us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:18:27.632401 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 22:18:27.632549 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 22:18:27.632715 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":412,"threads_started":1}
W20260811 22:18:27.634198 2113 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 22:18:29.238456 1122 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:18:29.238709 1122 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:18:29.242965 1122 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1ea83c38344487f8c0a8f0eff160c51: Sending DeleteTablet for 3 replicas of tablet 8d5c24a61f314d57a7591b50209c21ce
I20260811 22:18:29.243870 1360 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d5c24a61f314d57a7591b50209c21ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:29 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:58736
I20260811 22:18:29.243909 1611 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d5c24a61f314d57a7591b50209c21ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:29 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:57330
I20260811 22:18:29.243995 1862 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d5c24a61f314d57a7591b50209c21ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:29 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:51788
I20260811 22:18:29.244637 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 1243
I20260811 22:18:29.256333 2286 tablet_replica.cc:330] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685: stopping tablet replica
I20260811 22:18:29.256333 2284 tablet_replica.cc:330] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3: stopping tablet replica
I20260811 22:18:29.256614 2284 raft_consensus.cc:2229] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:18:29.256827 2286 raft_consensus.cc:2229] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:18:29.257071 2284 raft_consensus.cc:2258] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:29.257190 2286 raft_consensus.cc:2258] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:29.258354 2284 ts_tablet_manager.cc:1905] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:29.258410 2286 ts_tablet_manager.cc:1905] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:29.261055 2285 tablet_replica.cc:330] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320: stopping tablet replica
I20260811 22:18:29.261480 2285 raft_consensus.cc:2229] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:18:29.262275 2285 raft_consensus.cc:2258] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:29.264001 2285 ts_tablet_manager.cc:1905] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:29.265645 2286 ts_tablet_manager.cc:1918] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:18:29.265905 2286 log.cc:1196] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/8d5c24a61f314d57a7591b50209c21ce
I20260811 22:18:29.266451 2286 ts_tablet_manager.cc:1939] T 8d5c24a61f314d57a7591b50209c21ce P 84bfbf7596a846f0a69c8c26259f7685: Deleting consensus metadata
I20260811 22:18:29.266588 2284 ts_tablet_manager.cc:1918] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:18:29.266817 2284 log.cc:1196] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-0/wal/wals/8d5c24a61f314d57a7591b50209c21ce
I20260811 22:18:29.267340 2284 ts_tablet_manager.cc:1939] T 8d5c24a61f314d57a7591b50209c21ce P 9a977e616b4c41bf8dfe52ed84e655d3: Deleting consensus metadata
I20260811 22:18:29.267966 1098 catalog_manager.cc:4841] TS 84bfbf7596a846f0a69c8c26259f7685 (127.29.229.3:34577): tablet 8d5c24a61f314d57a7591b50209c21ce (table test-table [id=7c0b19e2736d43229ee9e005661c587f]) successfully deleted
I20260811 22:18:29.268463 1098 catalog_manager.cc:4841] TS 9a977e616b4c41bf8dfe52ed84e655d3 (127.29.229.1:35515): tablet 8d5c24a61f314d57a7591b50209c21ce (table test-table [id=7c0b19e2736d43229ee9e005661c587f]) successfully deleted
I20260811 22:18:29.271840 2285 ts_tablet_manager.cc:1918] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:18:29.272006 2285 log.cc:1196] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/8d5c24a61f314d57a7591b50209c21ce
I20260811 22:18:29.272572 2285 ts_tablet_manager.cc:1939] T 8d5c24a61f314d57a7591b50209c21ce P 3d16b93155a24aefa9a585715d61d320: Deleting consensus metadata
I20260811 22:18:29.273624 1100 catalog_manager.cc:4841] TS 3d16b93155a24aefa9a585715d61d320 (127.29.229.2:40199): tablet 8d5c24a61f314d57a7591b50209c21ce (table test-table [id=7c0b19e2736d43229ee9e005661c587f]) successfully deleted
Aug 11 22:18:29 dist-test-slave-jq4t krb5kdc[1052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486706, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:18:29.288395 2289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:29.248669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43972 (local address 127.29.229.1:35515)
0811 22:18:29.263088 (+ 14419us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:29.263098 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:29.266647 (+ 3549us) server_negotiation.cc:407] Connection header received
0811 22:18:29.266720 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:29.266727 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:29.266814 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:29.266951 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:29.268792 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:29.269548 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:29.271428 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:29.271948 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:29.272110 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:29.282595 (+ 10485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:29.282614 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:29.282631 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:29.282673 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:29.285828 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:29.286520 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:29.286527 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:29.286529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:29.286673 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:29.287053 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:29.287056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:29.287058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:29.287427 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:29.287652 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:29.287923 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 22:18:29.288105 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14207,"thread_start_us":14039,"threads_started":1}
W20260811 22:18:29.289312 1487 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43972
W20260811 22:18:29.290055 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 1243: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:18:29.290295 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 1243
I20260811 22:18:29.328806 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 1505
Aug 11 22:18:29 dist-test-slave-jq4t krb5kdc[1052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486706, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.229.2@KRBTEST.COM
I20260811 22:18:29.344816 2290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:29.329584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38000 (local address 127.29.229.2:40199)
0811 22:18:29.330118 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:29.330122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:29.330140 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:18:29.330198 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:29.330202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:29.330262 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:29.330364 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:29.331722 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:29.332652 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:29.333953 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:29.334354 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:29.334460 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:29.338936 (+ 4476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:29.338953 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:29.338971 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:29.339013 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:29.342158 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:29.342839 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:29.342847 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:29.342849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:29.342995 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:29.343339 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:29.343342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:29.343344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:29.343689 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:29.343943 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:29.344182 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 22:18:29.344370 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":274,"threads_started":1}
W20260811 22:18:29.345582 1726 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38000
W20260811 22:18:29.346278 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 1505: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:18:29.346470 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 1505
I20260811 22:18:29.392038 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 1742
Aug 11 22:18:29 dist-test-slave-jq4t krb5kdc[1052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486706, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.229.3@KRBTEST.COM
I20260811 22:18:29.410149 2291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:29.393057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40980 (local address 127.29.229.3:34577)
0811 22:18:29.393638 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:29.393645 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:29.393668 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:18:29.393740 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:29.393744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:29.393806 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:29.393933 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:29.395632 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:29.396428 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:29.398052 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:29.398468 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:29.398584 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:29.403078 (+ 4494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:29.403094 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:29.403110 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:29.403152 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:29.406797 (+ 3645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:29.407541 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:29.407549 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:29.407551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:29.407692 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:29.408288 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:29.408295 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:29.408298 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:29.408994 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:29.409252 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:29.409636 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 22:18:29.409841 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":283,"threads_started":1}
W20260811 22:18:29.410871 1965 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40980
W20260811 22:18:29.411501 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 1742: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:18:29.411676 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 1742
I20260811 22:18:29.451640 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 1070
I20260811 22:18:29.452960 1185 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:29.578737 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 1070
2026-08-11T22:18:29Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9202 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:18:29 dist-test-slave-jq4t krb5kdc[2299](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:18:29 dist-test-slave-jq4t krb5kdc[2299](info): set up 2 sockets
Aug 11 22:18:29 dist-test-slave-jq4t krb5kdc[2299](info): commencing operation
krb5kdc: starting...
W20260811 22:18:30.978042 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.338s user 0.005s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:18:31 dist-test-slave-jq4t krb5kdc[2299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:18:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:18:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:18:31.253414 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:37821
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:33375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:37821
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:31.607095 2322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:31.607398 2322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:31.607579 2322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:31.614930 2322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:31.615053 2322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:31.615144 2322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:31.615239 2322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:31.615324 2322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:18:31.625394 2322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:33375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:37821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:37821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:31.626776 2322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:31.628114 2322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:31.640893 2343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:31.640993 2341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:31.640964 2345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:31.641487 2322 server_base.cc:1034] running on GCE node
I20260811 22:18:31.642540 2322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:31.644007 2322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:31.645229 2322 hybrid_clock.cc:648] HybridClock initialized: now 1786486711645189 us; error 31 us; skew 500 ppm
I20260811 22:18:31.645550 2322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:31 dist-test-slave-jq4t krb5kdc[2299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486711, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:31.648867 2322 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:18:31.650410 2322 webserver.cc:460] Webserver started at http://127.29.229.62:41289/ using document root <none> and password file <none>
I20260811 22:18:31.651111 2322 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:31.651214 2322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:31.651502 2322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:31.654546 2322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "3dae50dcfbea4b76a2ac3603159124d1"
format_stamp: "Formatted at 2026-08-11 22:18:31 on dist-test-slave-jq4t"
server_key: "168b8a5cfce79c37799f8d6e9ee1e5d6"
server_key_iv: "999f87af13b27daba4fb4c7547bae0e9"
server_key_version: "encryptionkey@0"
I20260811 22:18:31.655284 2322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "3dae50dcfbea4b76a2ac3603159124d1"
format_stamp: "Formatted at 2026-08-11 22:18:31 on dist-test-slave-jq4t"
server_key: "168b8a5cfce79c37799f8d6e9ee1e5d6"
server_key_iv: "999f87af13b27daba4fb4c7547bae0e9"
server_key_version: "encryptionkey@0"
I20260811 22:18:31.660899 2322 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 22:18:31.664664 2356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:31.666083 2322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:18:31.666294 2322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "3dae50dcfbea4b76a2ac3603159124d1"
format_stamp: "Formatted at 2026-08-11 22:18:31 on dist-test-slave-jq4t"
server_key: "168b8a5cfce79c37799f8d6e9ee1e5d6"
server_key_iv: "999f87af13b27daba4fb4c7547bae0e9"
server_key_version: "encryptionkey@0"
I20260811 22:18:31.666515 2322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:31.681799 2322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:31.687844 2322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:31.688225 2322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:31.710098 2322 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:37821
I20260811 22:18:31.710117 2446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:37821 every 8 connection(s)
I20260811 22:18:31.711651 2322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:31.715456 2448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:31.721035 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 2322
I20260811 22:18:31.721441 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:18:31.722512 30612 external_mini_cluster.cc:1428] Setting key 3ca1a076d6cdb61d53b5a744b4cbcffc
I20260811 22:18:31.728372 2448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1: Bootstrap starting.
I20260811 22:18:31.733547 2448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:31.734984 2448 log.cc:824] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:31.738758 2448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1: No bootstrap required, opened a new log
Aug 11 22:18:31 dist-test-slave-jq4t krb5kdc[2299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:31.747605 2448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dae50dcfbea4b76a2ac3603159124d1" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37821 } }
I20260811 22:18:31.748046 2448 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:31.748200 2448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dae50dcfbea4b76a2ac3603159124d1, State: Initialized, Role: FOLLOWER
I20260811 22:18:31.748945 2448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dae50dcfbea4b76a2ac3603159124d1" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37821 } }
I20260811 22:18:31.749233 2448 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:18:31.749413 2448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:18:31.749612 2448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:31.750106 2455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:31.724833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48832 (local address 127.29.229.62:37821)
0811 22:18:31.725819 (+ 986us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:31.725833 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:31.725878 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:18:31.727781 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:31.727806 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:31.728227 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:31.728740 (+ 513us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:31.731370 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:31.732993 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:31.734988 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:31.736061 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:31.736287 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:31.741113 (+ 4826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:31.741120 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:31.741138 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:31.741177 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:31.745213 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:31.746117 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:31.746125 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:31.746132 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:31.746333 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:31.746749 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:31.746756 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:31.746767 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:31.747390 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:31.747715 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:31.748229 (+ 514us) server_negotiation.cc:299] Negotiation successful
0811 22:18:31.748798 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":506,"threads_started":1}
I20260811 22:18:31.752166 2448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dae50dcfbea4b76a2ac3603159124d1" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37821 } }
I20260811 22:18:31.752782 2448 leader_election.cc:302] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3dae50dcfbea4b76a2ac3603159124d1; no voters:
I20260811 22:18:31.753440 2448 leader_election.cc:288] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:18:31.753597 2459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:31.754318 2459 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1 [term 1 LEADER]: Becoming Leader. State: Replica: 3dae50dcfbea4b76a2ac3603159124d1, State: Running, Role: LEADER
I20260811 22:18:31.754827 2459 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dae50dcfbea4b76a2ac3603159124d1" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37821 } }
I20260811 22:18:31.755600 2448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:31.758373 2462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3dae50dcfbea4b76a2ac3603159124d1. Latest consensus state: current_term: 1 leader_uuid: "3dae50dcfbea4b76a2ac3603159124d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dae50dcfbea4b76a2ac3603159124d1" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37821 } } }
I20260811 22:18:31.758864 2462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:31.760414 2460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3dae50dcfbea4b76a2ac3603159124d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dae50dcfbea4b76a2ac3603159124d1" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37821 } } }
I20260811 22:18:31.760932 2460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:31.762398 2477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:31.767333 2477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:31.776871 2477 catalog_manager.cc:1347] Generated new cluster ID: 98955e71f448410cad0261f05c97ee56
I20260811 22:18:31.777019 2477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:31.794951 2477 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:18:31.796981 2477 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:31.809376 2477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3dae50dcfbea4b76a2ac3603159124d1: Generated new TSK 0
I20260811 22:18:31.810598 2477 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
I20260811 22:18:31.880790 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:37821
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:33375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:32.238049 2499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:32.238371 2499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:32.238549 2499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:32.245863 2499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:32.246016 2499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:32.246217 2499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:18:32.256772 2499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:33375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:37821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:32.258147 2499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:32.259646 2499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:32.273216 2508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:32.273172 2511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:32.273516 2507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:32.274448 2499 server_base.cc:1034] running on GCE node
I20260811 22:18:32.275094 2499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:32.276157 2499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:32.277446 2499 hybrid_clock.cc:648] HybridClock initialized: now 1786486712277375 us; error 63 us; skew 500 ppm
I20260811 22:18:32.277747 2499 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:32 dist-test-slave-jq4t krb5kdc[2299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486712, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:32.281319 2499 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:18:32.282936 2499 webserver.cc:460] Webserver started at http://127.29.229.1:34969/ using document root <none> and password file <none>
I20260811 22:18:32.283661 2499 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:32.283798 2499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:32.284150 2499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:32.287240 2499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "bf2b9e70d6e44e429722f7deca7e1e4a"
format_stamp: "Formatted at 2026-08-11 22:18:32 on dist-test-slave-jq4t"
server_key: "271ec8c44e60749321c293956671e911"
server_key_iv: "81116d074aa50bec891fdfbbc69cb625"
server_key_version: "encryptionkey@0"
I20260811 22:18:32.288069 2499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "bf2b9e70d6e44e429722f7deca7e1e4a"
format_stamp: "Formatted at 2026-08-11 22:18:32 on dist-test-slave-jq4t"
server_key: "271ec8c44e60749321c293956671e911"
server_key_iv: "81116d074aa50bec891fdfbbc69cb625"
server_key_version: "encryptionkey@0"
I20260811 22:18:32.297582 2499 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.009s sys 0.000s
I20260811 22:18:32.301932 2521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:32.303714 2499 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260811 22:18:32.303905 2499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "bf2b9e70d6e44e429722f7deca7e1e4a"
format_stamp: "Formatted at 2026-08-11 22:18:32 on dist-test-slave-jq4t"
server_key: "271ec8c44e60749321c293956671e911"
server_key_iv: "81116d074aa50bec891fdfbbc69cb625"
server_key_version: "encryptionkey@0"
I20260811 22:18:32.304188 2499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:32.320068 2499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:32.325840 2499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:32.326192 2499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:32.327229 2499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:32.328753 2499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:32.328856 2499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:32.329010 2499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:32.329090 2499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:32.369493 2499 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:33985
I20260811 22:18:32.369694 2728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:33985 every 8 connection(s)
I20260811 22:18:32.371075 2499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:18:32.378968 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 2499
I20260811 22:18:32.379365 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:18:32.380373 30612 external_mini_cluster.cc:1428] Setting key 0d34e2ee644a5eb90be8b9bf4c5bc33b
Aug 11 22:18:32 dist-test-slave-jq4t krb5kdc[2299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486712, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:32.395079 2732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:32.374836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:32981 (local address 127.29.229.62:37821)
0811 22:18:32.375350 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:32.375355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:32.376737 (+ 1382us) server_negotiation.cc:407] Connection header received
0811 22:18:32.378904 (+ 2167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:32.378911 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:32.379025 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:32.379156 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:32.381190 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:32.382231 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:32.384120 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:32.384606 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:32.384752 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:32.388794 (+ 4042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:32.388799 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:32.388803 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:32.388840 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:32.391597 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:32.392619 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:32.392626 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:32.392635 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:32.392757 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:32.393236 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:32.393240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:32.393242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:32.393523 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:32.393682 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:32.394431 (+ 749us) server_negotiation.cc:299] Negotiation successful
0811 22:18:32.394684 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":281,"threads_started":1}
I20260811 22:18:32.396111 2733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:32.375604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:37821 (local address 127.29.229.1:32981)
0811 22:18:32.376530 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:32.376601 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:32.378584 (+ 1983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:32.379557 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:32.379567 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:32.380250 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:32.380999 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:32.381019 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:32.382447 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:32.382451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:32.383954 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:32.383969 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:32.384857 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:32.384866 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:32.385123 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:32.385884 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:32.385909 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:32.388530 (+ 2621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:32.391776 (+ 3246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:32.391786 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:32.391804 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:32.392430 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:32.392878 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:32.392883 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:32.392887 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:32.393103 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:32.393656 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:32.393664 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:32.394043 (+ 379us) client_negotiation.cc:769] Sending connection context
0811 22:18:32.394309 (+ 266us) client_negotiation.cc:240] Negotiation successful
0811 22:18:32.394882 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":330,"threads_started":1}
I20260811 22:18:32.398167 2730 heartbeater.cc:347] Connected to a master server at 127.29.229.62:37821
I20260811 22:18:32.398546 2730 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:32.399503 2730 heartbeater.cc:511] Master 127.29.229.62:37821 requested a full tablet report, sending...
I20260811 22:18:32.402057 2383 ts_manager.cc:194] Registered new tserver with Master: bf2b9e70d6e44e429722f7deca7e1e4a (127.29.229.1:33985)
I20260811 22:18:32.403553 2383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:32981
WARNING: no policy specified for kudu/127.29.229.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
I20260811 22:18:32.440376 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:37821
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:33375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:32.804790 2738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:32.805083 2738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:32.805187 2738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:32.813436 2738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:32.813567 2738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:32.813684 2738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:18:32.823686 2738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:33375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:37821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:32.825099 2738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:32.826368 2738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:32.840664 2753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:32.840595 2751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:32.840763 2758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:32.841802 2738 server_base.cc:1034] running on GCE node
I20260811 22:18:32.842449 2738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:32.843464 2738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:32.844692 2738 hybrid_clock.cc:648] HybridClock initialized: now 1786486712844649 us; error 62 us; skew 500 ppm
I20260811 22:18:32.845013 2738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:32 dist-test-slave-jq4t krb5kdc[2299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486712, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:32.848301 2738 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:18:32.849814 2738 webserver.cc:460] Webserver started at http://127.29.229.2:34439/ using document root <none> and password file <none>
I20260811 22:18:32.850476 2738 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:32.850581 2738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:32.850858 2738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:32.853950 2738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "9864c123e4044c6a8ce787d830ea82a5"
format_stamp: "Formatted at 2026-08-11 22:18:32 on dist-test-slave-jq4t"
server_key: "20b93b4690605255cf2738797ffe2e01"
server_key_iv: "9299237c4ba344829d9b6c61dbaffcb7"
server_key_version: "encryptionkey@0"
I20260811 22:18:32.854722 2738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "9864c123e4044c6a8ce787d830ea82a5"
format_stamp: "Formatted at 2026-08-11 22:18:32 on dist-test-slave-jq4t"
server_key: "20b93b4690605255cf2738797ffe2e01"
server_key_iv: "9299237c4ba344829d9b6c61dbaffcb7"
server_key_version: "encryptionkey@0"
I20260811 22:18:32.860288 2738 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 22:18:32.863972 2767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:32.865589 2738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:18:32.865788 2738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "9864c123e4044c6a8ce787d830ea82a5"
format_stamp: "Formatted at 2026-08-11 22:18:32 on dist-test-slave-jq4t"
server_key: "20b93b4690605255cf2738797ffe2e01"
server_key_iv: "9299237c4ba344829d9b6c61dbaffcb7"
server_key_version: "encryptionkey@0"
I20260811 22:18:32.866070 2738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:32.886988 2738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:32.891625 2738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:32.891896 2738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:32.892968 2738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:32.894552 2738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:32.894634 2738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:32.894723 2738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:32.894768 2738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:32.930402 2738 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:46809
I20260811 22:18:32.930509 2960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:46809 every 8 connection(s)
I20260811 22:18:32.931751 2738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:18:32.938917 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 2738
I20260811 22:18:32.939131 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:18:32.939575 30612 external_mini_cluster.cc:1428] Setting key 0a93116cba4a787fe50d125355d4042b
Aug 11 22:18:32 dist-test-slave-jq4t krb5kdc[2299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486712, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:32.954681 2968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:32.934647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:39843 (local address 127.29.229.62:37821)
0811 22:18:32.935174 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:32.935181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:32.936104 (+ 923us) server_negotiation.cc:407] Connection header received
0811 22:18:32.937964 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:32.937968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:32.938037 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:32.938137 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:32.939678 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:32.940763 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:32.942738 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:32.943523 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:32.943675 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:32.947279 (+ 3604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:32.947288 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:32.947295 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:32.947340 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:32.951023 (+ 3683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:32.952205 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:32.952209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:32.952211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:32.952339 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:32.952886 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:32.952889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:32.952891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:32.953169 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:32.953283 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:32.954221 (+ 938us) server_negotiation.cc:299] Negotiation successful
0811 22:18:32.954395 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":252,"threads_started":1}
I20260811 22:18:32.956368 2969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:32.935156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:37821 (local address 127.29.229.2:39843)
0811 22:18:32.935926 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:32.935965 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:32.937705 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:32.938339 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:32.938347 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:32.938850 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:32.939507 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:32.939524 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:32.940920 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:32.940925 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:32.942579 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:32.942593 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:32.943771 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:32.943776 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:32.944009 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:32.944786 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:32.944815 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:32.947033 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:32.951312 (+ 4279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:32.951325 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:32.951346 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:32.952080 (+ 734us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:32.952457 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:32.952463 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:32.952467 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:32.952773 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:32.953306 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:32.953318 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:32.953814 (+ 496us) client_negotiation.cc:769] Sending connection context
0811 22:18:32.954143 (+ 329us) client_negotiation.cc:240] Negotiation successful
0811 22:18:32.954767 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":293,"threads_started":1}
I20260811 22:18:32.958045 2961 heartbeater.cc:347] Connected to a master server at 127.29.229.62:37821
I20260811 22:18:32.958377 2961 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:32.959220 2961 heartbeater.cc:511] Master 127.29.229.62:37821 requested a full tablet report, sending...
I20260811 22:18:32.960891 2383 ts_manager.cc:194] Registered new tserver with Master: 9864c123e4044c6a8ce787d830ea82a5 (127.29.229.2:46809)
I20260811 22:18:32.962220 2383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:39843
WARNING: no policy specified for kudu/127.29.229.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
I20260811 22:18:32.998122 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:37821
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:33375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:33.353552 2974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:33.353912 2974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:33.354094 2974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:33.361769 2974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:33.361933 2974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:33.362116 2974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:18:33.375476 2974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:33375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:37821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:33.376878 2974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:33.378163 2974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:33.394560 2994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:33.394495 2990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:33.394605 2996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:33.395349 2974 server_base.cc:1034] running on GCE node
I20260811 22:18:33.396413 2974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:33.397569 2974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:33.398809 2974 hybrid_clock.cc:648] HybridClock initialized: now 1786486713398747 us; error 56 us; skew 500 ppm
I20260811 22:18:33.399169 2974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:33 dist-test-slave-jq4t krb5kdc[2299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486713, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:33.402602 2974 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:18:33.404196 2974 webserver.cc:460] Webserver started at http://127.29.229.3:37587/ using document root <none> and password file <none>
I20260811 22:18:33.404960 2974 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:33.405079 2974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:33.405354 2974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:33.407511 2730 heartbeater.cc:503] Master 127.29.229.62:37821 was elected leader, sending a full tablet report...
I20260811 22:18:33.408782 2974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "eb98b23093704a06b0db0c520b0b6b1d"
format_stamp: "Formatted at 2026-08-11 22:18:33 on dist-test-slave-jq4t"
server_key: "e6afe6e58adb0d55ec3a7c7940ce38be"
server_key_iv: "a148a6ab7340642aa900c1e84cc3aef0"
server_key_version: "encryptionkey@0"
I20260811 22:18:33.409688 2974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "eb98b23093704a06b0db0c520b0b6b1d"
format_stamp: "Formatted at 2026-08-11 22:18:33 on dist-test-slave-jq4t"
server_key: "e6afe6e58adb0d55ec3a7c7940ce38be"
server_key_iv: "a148a6ab7340642aa900c1e84cc3aef0"
server_key_version: "encryptionkey@0"
I20260811 22:18:33.415261 2974 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 22:18:33.418952 3008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:33.420361 2974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:18:33.420590 2974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "eb98b23093704a06b0db0c520b0b6b1d"
format_stamp: "Formatted at 2026-08-11 22:18:33 on dist-test-slave-jq4t"
server_key: "e6afe6e58adb0d55ec3a7c7940ce38be"
server_key_iv: "a148a6ab7340642aa900c1e84cc3aef0"
server_key_version: "encryptionkey@0"
I20260811 22:18:33.420812 2974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:33.444154 2974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:33.449100 2974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:33.449433 2974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:33.450556 2974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:33.452064 2974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:33.452174 2974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:33.452333 2974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:33.452412 2974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:33.494161 2974 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:33875
I20260811 22:18:33.494232 3228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:33875 every 8 connection(s)
I20260811 22:18:33.495792 2974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:18:33.505869 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 2974
I20260811 22:18:33.506104 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:18:33.506569 30612 external_mini_cluster.cc:1428] Setting key cc85cccfa0f1277fc61056536ae41294
Aug 11 22:18:33 dist-test-slave-jq4t krb5kdc[2299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486713, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:33.518787 3240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:33.498894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:40727 (local address 127.29.229.62:37821)
0811 22:18:33.499548 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:33.499556 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:33.500590 (+ 1034us) server_negotiation.cc:407] Connection header received
0811 22:18:33.502752 (+ 2162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:33.502756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:33.502835 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:33.502943 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:33.504733 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:33.505554 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:33.507356 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:33.507955 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:33.508099 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:33.512185 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:33.512193 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:33.512200 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:33.512241 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:33.515291 (+ 3050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:33.516165 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:33.516170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:33.516172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:33.516301 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:33.516851 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:33.516860 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:33.516862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:33.517150 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:33.517325 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:33.518112 (+ 787us) server_negotiation.cc:299] Negotiation successful
0811 22:18:33.518363 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":292,"threads_started":1}
I20260811 22:18:33.519795 3242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:33.499292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:37821 (local address 127.29.229.3:40727)
0811 22:18:33.500369 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:33.500419 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:33.502397 (+ 1978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:33.503230 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:33.503239 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:33.503822 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:33.504443 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:33.504468 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:33.505734 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:33.505740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:33.507200 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:33.507214 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:33.508206 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:33.508214 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:33.508541 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:33.509450 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:33.509485 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:33.511942 (+ 2457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:33.515476 (+ 3534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:33.515486 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:33.515506 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:33.516016 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:33.516424 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:33.516429 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:33.516433 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:33.516721 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:33.517291 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:33.517300 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:33.517704 (+ 404us) client_negotiation.cc:769] Sending connection context
0811 22:18:33.518001 (+ 297us) client_negotiation.cc:240] Negotiation successful
0811 22:18:33.518585 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"thread_start_us":550,"threads_started":1}
I20260811 22:18:33.521456 3234 heartbeater.cc:347] Connected to a master server at 127.29.229.62:37821
I20260811 22:18:33.521853 3234 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:33.522784 3234 heartbeater.cc:511] Master 127.29.229.62:37821 requested a full tablet report, sending...
I20260811 22:18:33.524446 2383 ts_manager.cc:194] Registered new tserver with Master: eb98b23093704a06b0db0c520b0b6b1d (127.29.229.3:33875)
I20260811 22:18:33.525398 2383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:40727
I20260811 22:18:33.536652 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:18:33.582472 3240 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:18:33.579658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48842 (local address 127.29.229.62:37821)
0811 22:18:33.579927 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:33.579935 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:33.580218 (+ 283us) server_negotiation.cc:407] Connection header received
0811 22:18:33.580370 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:33.580374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:33.580450 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:33.580635 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:33.582143 (+ 1508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48842: BlockingRecv error: recv got EOF from 127.0.0.1:48842 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 22:18:33.585717 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 2499
W20260811 22:18:33.589340 3257 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:18:33.586981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34014 (local address 127.29.229.1:33985)
0811 22:18:33.587630 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:33.587637 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:33.587660 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:18:33.587720 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:33.587726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:33.587934 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:33.588112 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:33.589005 (+ 893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34014: BlockingRecv error: recv got EOF from 127.0.0.1:34014 (error 108)
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":324,"threads_started":1}
W20260811 22:18:33.589345 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 2499: Not authorized: Client connection negotiation failed: client connection to 127.29.229.1:33985: server requires authentication, but client does not have Kerberos credentials available
I20260811 22:18:33.589668 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 2499
I20260811 22:18:33.622246 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 2738
W20260811 22:18:33.625432 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 2738: Not authorized: Client connection negotiation failed: client connection to 127.29.229.2:46809: server requires authentication, but client does not have Kerberos credentials available
I20260811 22:18:33.625641 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 2738
W20260811 22:18:33.625728 3258 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:18:33.623401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33206 (local address 127.29.229.2:46809)
0811 22:18:33.623973 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:33.623981 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:33.624006 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:18:33.624079 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:33.624085 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:33.624293 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:33.624489 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:33.625351 (+ 862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33206: BlockingRecv error: recv got EOF from 127.0.0.1:33206 (error 108)
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":298,"threads_started":1}
I20260811 22:18:33.656333 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 2974
W20260811 22:18:33.659624 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 2974: Not authorized: Client connection negotiation failed: client connection to 127.29.229.3:33875: server requires authentication, but client does not have Kerberos credentials available
I20260811 22:18:33.659793 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 2974
I20260811 22:18:33.686154 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 2322
I20260811 22:18:33.687574 2442 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:33.811501 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 2322
2026-08-11T22:18:33Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4227 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:18:33 dist-test-slave-jq4t krb5kdc[3266](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:18:33 dist-test-slave-jq4t krb5kdc[3266](info): set up 2 sockets
Aug 11 22:18:33 dist-test-slave-jq4t krb5kdc[3266](info): commencing operation
krb5kdc: starting...
W20260811 22:18:35.220212 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.353s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:18:35 dist-test-slave-jq4t krb5kdc[3266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:18:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:18:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:18:35.501754 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:40385
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:37297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:40385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:35.876878 3289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:35.877333 3289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:35.877533 3289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:35.885380 3289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:35.885540 3289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:35.885661 3289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:35.885766 3289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:35.885859 3289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:18:35.896301 3289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:37297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:40385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:40385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:35.897789 3289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:35.899161 3289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:35.912587 3297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:35.912617 3299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:35.912866 3295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:35.913607 3289 server_base.cc:1034] running on GCE node
I20260811 22:18:35.914562 3289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:35.916152 3289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:35.917435 3289 hybrid_clock.cc:648] HybridClock initialized: now 1786486715917346 us; error 72 us; skew 500 ppm
I20260811 22:18:35.917845 3289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:35 dist-test-slave-jq4t krb5kdc[3266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486715, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:35.921389 3289 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:18:35.923043 3289 webserver.cc:460] Webserver started at http://127.29.229.62:38027/ using document root <none> and password file <none>
I20260811 22:18:35.923796 3289 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:35.924162 3289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:35.924589 3289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:35.929049 3289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "a7e384c489bc40b0a69288e3d59c0f94"
format_stamp: "Formatted at 2026-08-11 22:18:35 on dist-test-slave-jq4t"
server_key: "eb50d0ae6119940167c7d3915f5ca4ae"
server_key_iv: "7e01da755c6ce85b2abcc596906fe99f"
server_key_version: "encryptionkey@0"
I20260811 22:18:35.930380 3289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "a7e384c489bc40b0a69288e3d59c0f94"
format_stamp: "Formatted at 2026-08-11 22:18:35 on dist-test-slave-jq4t"
server_key: "eb50d0ae6119940167c7d3915f5ca4ae"
server_key_iv: "7e01da755c6ce85b2abcc596906fe99f"
server_key_version: "encryptionkey@0"
I20260811 22:18:35.936234 3289 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 22:18:35.940037 3313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:35.941953 3289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:18:35.942174 3289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "a7e384c489bc40b0a69288e3d59c0f94"
format_stamp: "Formatted at 2026-08-11 22:18:35 on dist-test-slave-jq4t"
server_key: "eb50d0ae6119940167c7d3915f5ca4ae"
server_key_iv: "7e01da755c6ce85b2abcc596906fe99f"
server_key_version: "encryptionkey@0"
I20260811 22:18:35.942437 3289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:35.960300 3289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:35.965111 3289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:35.965444 3289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:35.987812 3289 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:40385
I20260811 22:18:35.987871 3406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:40385 every 8 connection(s)
I20260811 22:18:35.989459 3289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:35.992982 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 3289
I20260811 22:18:35.993254 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:18:35.994158 30612 external_mini_cluster.cc:1428] Setting key c17afa844b33be2b4dedf9bb75768e84
I20260811 22:18:35.994155 3407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 22:18:36 dist-test-slave-jq4t krb5kdc[3266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:36.012409 3407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94: Bootstrap starting.
I20260811 22:18:36.017863 3407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:36.019302 3407 log.cc:824] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:36.022636 3407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94: No bootstrap required, opened a new log
I20260811 22:18:36.022665 3410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:35.996253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38800 (local address 127.29.229.62:40385)
0811 22:18:35.997199 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:35.997211 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:35.997254 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:18:35.999540 (+ 2286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:35.999567 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:36.000009 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:36.000394 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:36.002639 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:36.004381 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:36.006556 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:36.007512 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:36.007703 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:36.013368 (+ 5665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:36.013378 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:36.013396 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:36.013441 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:36.017785 (+ 4344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:36.018878 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:36.018887 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:36.018901 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:36.019145 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:36.019594 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:36.019599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:36.019604 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:36.020289 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:36.020670 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:36.021157 (+ 487us) server_negotiation.cc:299] Negotiation successful
0811 22:18:36.021646 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":473,"threads_started":1}
I20260811 22:18:36.027541 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:18:36.034883 3407 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e384c489bc40b0a69288e3d59c0f94" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40385 } }
I20260811 22:18:36.035372 3407 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:36.035497 3407 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e384c489bc40b0a69288e3d59c0f94, State: Initialized, Role: FOLLOWER
I20260811 22:18:36.036132 3407 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e384c489bc40b0a69288e3d59c0f94" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40385 } }
I20260811 22:18:36.036397 3407 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:18:36.036581 3407 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:18:36.036760 3407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:36.039081 3407 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e384c489bc40b0a69288e3d59c0f94" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40385 } }
I20260811 22:18:36.039644 3407 leader_election.cc:302] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7e384c489bc40b0a69288e3d59c0f94; no voters:
I20260811 22:18:36.040373 3407 leader_election.cc:288] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:18:36.040505 3414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:36.041924 3414 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94 [term 1 LEADER]: Becoming Leader. State: Replica: a7e384c489bc40b0a69288e3d59c0f94, State: Running, Role: LEADER
I20260811 22:18:36.042594 3414 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e384c489bc40b0a69288e3d59c0f94" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40385 } }
I20260811 22:18:36.042995 3407 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:36.045837 3425 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7e384c489bc40b0a69288e3d59c0f94. Latest consensus state: current_term: 1 leader_uuid: "a7e384c489bc40b0a69288e3d59c0f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e384c489bc40b0a69288e3d59c0f94" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40385 } } }
I20260811 22:18:36.046367 3425 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:36.047060 3416 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7e384c489bc40b0a69288e3d59c0f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e384c489bc40b0a69288e3d59c0f94" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40385 } } }
I20260811 22:18:36.047421 3416 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:36.066126 3430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:36.073055 3430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:36.084133 3430 catalog_manager.cc:1347] Generated new cluster ID: 1a99233b6d1649f290601d2fede2b368
I20260811 22:18:36.084283 3430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:36.099781 3410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:36.087212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38812 (local address 127.29.229.62:40385)
0811 22:18:36.087407 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:36.087414 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:36.089153 (+ 1739us) server_negotiation.cc:407] Connection header received
0811 22:18:36.089350 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:36.089356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:36.089448 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:36.089559 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:36.090912 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:36.092401 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:36.094898 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:36.095639 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:36.095789 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:36.097706 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:36.097713 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:36.097719 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:18:36.098732 (+ 1013us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:36.098799 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:18:36.098840 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:36.098934 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:36.099227 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 22:18:36.099363 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:18:36.105877 3430 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:18:36.107240 3430 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:36.119845 3430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7e384c489bc40b0a69288e3d59c0f94: Generated new TSK 0
I20260811 22:18:36.120863 3430 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:18:36.134225 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 3289
I20260811 22:18:36.135600 3399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:36.232604 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 3289
2026-08-11T22:18:36Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2417 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:18:36 dist-test-slave-jq4t krb5kdc[3475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:18:36 dist-test-slave-jq4t krb5kdc[3475](info): set up 2 sockets
Aug 11 22:18:36 dist-test-slave-jq4t krb5kdc[3475](info): commencing operation
krb5kdc: starting...
W20260811 22:18:37.610239 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.012s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:18:37 dist-test-slave-jq4t krb5kdc[3475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:18:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:18:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:18:37.885274 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:44257
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:44257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:38.243146 3502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:38.243469 3502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:38.243644 3502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:38.251231 3502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:38.251361 3502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:38.251449 3502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:38.251549 3502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:38.251632 3502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:18:38.261983 3502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:44257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:44257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:38.263420 3502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:38.264786 3502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:38.277206 3510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:38.277182 3513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:38.277240 3517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:38.278527 3502 server_base.cc:1034] running on GCE node
I20260811 22:18:38.279326 3502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:38.280831 3502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:38.282083 3502 hybrid_clock.cc:648] HybridClock initialized: now 1786486718282034 us; error 49 us; skew 500 ppm
I20260811 22:18:38.282431 3502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:38 dist-test-slave-jq4t krb5kdc[3475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486718, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:38.285710 3502 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:18:38.287309 3502 webserver.cc:460] Webserver started at http://127.29.229.62:46623/ using document root <none> and password file <none>
I20260811 22:18:38.288002 3502 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:38.288156 3502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:38.288539 3502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:38.291568 3502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "43f2da25f51c45638863d9077696c2a9"
format_stamp: "Formatted at 2026-08-11 22:18:38 on dist-test-slave-jq4t"
server_key: "0700b8203f2a447e26a83273b5664e32"
server_key_iv: "66c327ec1d178681a3a12d0dd2e7d76f"
server_key_version: "encryptionkey@0"
I20260811 22:18:38.292596 3502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "43f2da25f51c45638863d9077696c2a9"
format_stamp: "Formatted at 2026-08-11 22:18:38 on dist-test-slave-jq4t"
server_key: "0700b8203f2a447e26a83273b5664e32"
server_key_iv: "66c327ec1d178681a3a12d0dd2e7d76f"
server_key_version: "encryptionkey@0"
I20260811 22:18:38.298002 3502 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 22:18:38.301508 3530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:38.303073 3502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:18:38.303283 3502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "43f2da25f51c45638863d9077696c2a9"
format_stamp: "Formatted at 2026-08-11 22:18:38 on dist-test-slave-jq4t"
server_key: "0700b8203f2a447e26a83273b5664e32"
server_key_iv: "66c327ec1d178681a3a12d0dd2e7d76f"
server_key_version: "encryptionkey@0"
I20260811 22:18:38.303509 3502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:38.337020 3502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:38.342092 3502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:38.342424 3502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:38.364426 3502 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:44257
I20260811 22:18:38.364428 3632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:44257 every 8 connection(s)
I20260811 22:18:38.365935 3502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:38.369719 3633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:38.375844 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 3502
I20260811 22:18:38.376086 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:18:38.376983 30612 external_mini_cluster.cc:1428] Setting key 2d2a920a15006e540c8218599f4c6418
I20260811 22:18:38.383093 3633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9: Bootstrap starting.
I20260811 22:18:38.387988 3633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:38.389590 3633 log.cc:824] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:38.393520 3633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9: No bootstrap required, opened a new log
Aug 11 22:18:38 dist-test-slave-jq4t krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:38.406543 3633 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f2da25f51c45638863d9077696c2a9" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44257 } }
I20260811 22:18:38.407104 3633 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:38.407225 3633 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43f2da25f51c45638863d9077696c2a9, State: Initialized, Role: FOLLOWER
I20260811 22:18:38.408224 3633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f2da25f51c45638863d9077696c2a9" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44257 } }
I20260811 22:18:38.408576 3633 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:18:38.408737 3633 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:18:38.408973 3640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:38.379413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41248 (local address 127.29.229.62:44257)
0811 22:18:38.381022 (+ 1609us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:38.381038 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:38.381093 (+ 55us) server_negotiation.cc:407] Connection header received
0811 22:18:38.383999 (+ 2906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:38.384034 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:38.384611 (+ 577us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:38.385129 (+ 518us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:38.387008 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:38.389094 (+ 2086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:38.390776 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:38.391720 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:38.391965 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:38.397021 (+ 5056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:38.397038 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:38.397067 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:38.397133 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:38.401987 (+ 4854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:38.402926 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:38.402935 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:38.402945 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:38.403193 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:38.403775 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:38.403783 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:38.403788 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:38.404785 (+ 997us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:38.405153 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:38.406705 (+ 1552us) server_negotiation.cc:299] Negotiation successful
0811 22:18:38.407299 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1342,"thread_start_us":522,"threads_started":1}
I20260811 22:18:38.408979 3633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:38.412603 3633 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f2da25f51c45638863d9077696c2a9" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44257 } }
I20260811 22:18:38.413208 3633 leader_election.cc:302] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43f2da25f51c45638863d9077696c2a9; no voters:
I20260811 22:18:38.413818 3633 leader_election.cc:288] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:18:38.414019 3642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:38.414791 3642 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9 [term 1 LEADER]: Becoming Leader. State: Replica: 43f2da25f51c45638863d9077696c2a9, State: Running, Role: LEADER
I20260811 22:18:38.415364 3642 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f2da25f51c45638863d9077696c2a9" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44257 } }
I20260811 22:18:38.416091 3633 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:38.418735 3643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43f2da25f51c45638863d9077696c2a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f2da25f51c45638863d9077696c2a9" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44257 } } }
I20260811 22:18:38.419399 3643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:38.418723 3646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43f2da25f51c45638863d9077696c2a9. Latest consensus state: current_term: 1 leader_uuid: "43f2da25f51c45638863d9077696c2a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f2da25f51c45638863d9077696c2a9" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44257 } } }
I20260811 22:18:38.419785 3646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9 [sys.catalog]: This master's current role is: LEADER
W20260811 22:18:38.433332 3671 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:18:38.433491 3671 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:18:38.433645 3651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:38.435846 3651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:38.445369 3651 catalog_manager.cc:1347] Generated new cluster ID: fb9ad15067844682ab1b325abb86aff1
I20260811 22:18:38.445497 3651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:38.462584 3651 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:18:38.464560 3651 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:38.476608 3651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43f2da25f51c45638863d9077696c2a9: Generated new TSK 0
I20260811 22:18:38.477671 3651 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
I20260811 22:18:38.540967 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:44257
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:38.912700 3689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:38.913010 3689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:38.913117 3689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:38.921160 3689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:38.921284 3689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:38.921403 3689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:18:38.932274 3689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:44257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:38.933761 3689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:38.935165 3689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:38.949352 3699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:38.949831 3701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:38.949920 3704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:38.950285 3689 server_base.cc:1034] running on GCE node
I20260811 22:18:38.951040 3689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:38.952190 3689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:38.955596 3689 hybrid_clock.cc:648] HybridClock initialized: now 1786486718955525 us; error 81 us; skew 500 ppm
I20260811 22:18:38.955991 3689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:38 dist-test-slave-jq4t krb5kdc[3475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486718, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:38.959900 3689 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:18:38.961540 3689 webserver.cc:460] Webserver started at http://127.29.229.1:42893/ using document root <none> and password file <none>
I20260811 22:18:38.962255 3689 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:38.962368 3689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:38.962639 3689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:38.965776 3689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "85dfdb6190574476a852c2411a225b87"
format_stamp: "Formatted at 2026-08-11 22:18:38 on dist-test-slave-jq4t"
server_key: "223057b811f965a0ef052192c1ab6cb1"
server_key_iv: "ce0bae033bef605053924d8ad599c193"
server_key_version: "encryptionkey@0"
I20260811 22:18:38.966549 3689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "85dfdb6190574476a852c2411a225b87"
format_stamp: "Formatted at 2026-08-11 22:18:38 on dist-test-slave-jq4t"
server_key: "223057b811f965a0ef052192c1ab6cb1"
server_key_iv: "ce0bae033bef605053924d8ad599c193"
server_key_version: "encryptionkey@0"
I20260811 22:18:38.972360 3689 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 22:18:38.976402 3721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:38.978106 3689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:18:38.978259 3689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "85dfdb6190574476a852c2411a225b87"
format_stamp: "Formatted at 2026-08-11 22:18:38 on dist-test-slave-jq4t"
server_key: "223057b811f965a0ef052192c1ab6cb1"
server_key_iv: "ce0bae033bef605053924d8ad599c193"
server_key_version: "encryptionkey@0"
I20260811 22:18:38.978435 3689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:39.004385 3689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:39.010370 3689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:39.010752 3689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:39.011857 3689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:39.013340 3689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:39.013420 3689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:39.013509 3689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:39.013546 3689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:39.052784 3689 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:44489
I20260811 22:18:39.052845 3919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:44489 every 8 connection(s)
I20260811 22:18:39.054328 3689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:18:39.058737 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 3689
I20260811 22:18:39.059016 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:18:39.059872 30612 external_mini_cluster.cc:1428] Setting key 081a7d923bd34f8ac52f0bb8eb81469b
Aug 11 22:18:39 dist-test-slave-jq4t krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486718, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:39.077922 3929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:39.057854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:59861 (local address 127.29.229.62:44257)
0811 22:18:39.058361 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:39.058366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:39.059862 (+ 1496us) server_negotiation.cc:407] Connection header received
0811 22:18:39.062160 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:39.062165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:39.062259 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:39.062393 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:39.063952 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:39.064766 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:39.066355 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:39.066764 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:39.066900 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:39.070651 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:39.070657 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:39.070660 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:39.070693 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:39.073870 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:39.075295 (+ 1425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:39.075303 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:39.075307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:39.075469 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:39.075951 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:39.075958 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:39.075961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:39.076250 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:39.076422 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:39.077252 (+ 830us) server_negotiation.cc:299] Negotiation successful
0811 22:18:39.077520 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":293,"threads_started":1}
I20260811 22:18:39.078980 3932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:39.058440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:44257 (local address 127.29.229.1:59861)
0811 22:18:39.059691 (+ 1251us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:39.059738 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:39.061884 (+ 2146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:39.062647 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:39.062656 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:39.063180 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:39.063772 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:39.063789 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:39.064932 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:39.064937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:39.066189 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:39.066201 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:39.067002 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:39.067011 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:39.067248 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:39.067972 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:39.067996 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:39.070404 (+ 2408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:39.074306 (+ 3902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:39.074317 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:39.074338 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:39.075119 (+ 781us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:39.075602 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:39.075607 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:39.075618 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:39.075820 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:39.076416 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:39.076425 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:39.076856 (+ 431us) client_negotiation.cc:769] Sending connection context
0811 22:18:39.077153 (+ 297us) client_negotiation.cc:240] Negotiation successful
0811 22:18:39.077742 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"thread_start_us":450,"threads_started":1}
I20260811 22:18:39.080832 3921 heartbeater.cc:347] Connected to a master server at 127.29.229.62:44257
I20260811 22:18:39.081171 3921 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:39.082052 3921 heartbeater.cc:511] Master 127.29.229.62:44257 requested a full tablet report, sending...
I20260811 22:18:39.084934 3570 ts_manager.cc:194] Registered new tserver with Master: 85dfdb6190574476a852c2411a225b87 (127.29.229.1:44489)
I20260811 22:18:39.087219 3570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:59861
WARNING: no policy specified for kudu/127.29.229.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
I20260811 22:18:39.122326 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:44257
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:39.488561 3939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:39.488952 3939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:39.489064 3939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:39.496706 3939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:39.496831 3939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:39.496949 3939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:18:39.507074 3939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:44257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:39.508462 3939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:39.509877 3939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:39.525090 3952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:39.524978 3951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:39.525143 3955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:39.525765 3939 server_base.cc:1034] running on GCE node
I20260811 22:18:39.526482 3939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:39.527601 3939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:39.528820 3939 hybrid_clock.cc:648] HybridClock initialized: now 1786486719528737 us; error 61 us; skew 500 ppm
I20260811 22:18:39.529193 3939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:39 dist-test-slave-jq4t krb5kdc[3475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486719, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:39.532858 3939 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:18:39.534682 3939 webserver.cc:460] Webserver started at http://127.29.229.2:41109/ using document root <none> and password file <none>
I20260811 22:18:39.535434 3939 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:39.535574 3939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:39.535925 3939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:39.539059 3939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "9b730b4e966342ababa39ecbfbff14de"
format_stamp: "Formatted at 2026-08-11 22:18:39 on dist-test-slave-jq4t"
server_key: "d9291dae7b98c5f27c974d53264c7ca3"
server_key_iv: "71240787780d18e1c6120bdca5b76cc2"
server_key_version: "encryptionkey@0"
I20260811 22:18:39.539883 3939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "9b730b4e966342ababa39ecbfbff14de"
format_stamp: "Formatted at 2026-08-11 22:18:39 on dist-test-slave-jq4t"
server_key: "d9291dae7b98c5f27c974d53264c7ca3"
server_key_iv: "71240787780d18e1c6120bdca5b76cc2"
server_key_version: "encryptionkey@0"
I20260811 22:18:39.545442 3939 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 22:18:39.549274 3969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:39.550957 3939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:18:39.551160 3939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "9b730b4e966342ababa39ecbfbff14de"
format_stamp: "Formatted at 2026-08-11 22:18:39 on dist-test-slave-jq4t"
server_key: "d9291dae7b98c5f27c974d53264c7ca3"
server_key_iv: "71240787780d18e1c6120bdca5b76cc2"
server_key_version: "encryptionkey@0"
I20260811 22:18:39.551371 3939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:39.576095 3939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:39.581482 3939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:39.581866 3939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:39.583106 3939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:39.584944 3939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:39.585062 3939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:39.585162 3939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:39.585210 3939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:39.625753 3939 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:40967
I20260811 22:18:39.625771 4185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:40967 every 8 connection(s)
I20260811 22:18:39.627231 3939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:18:39.631918 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 3939
I20260811 22:18:39.632118 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:18:39.632814 30612 external_mini_cluster.cc:1428] Setting key f303378451b2efd856bd67790c665689
Aug 11 22:18:39 dist-test-slave-jq4t krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486719, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:39.652122 4192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:39.630045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:35367 (local address 127.29.229.62:44257)
0811 22:18:39.630554 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:39.630559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:39.631768 (+ 1209us) server_negotiation.cc:407] Connection header received
0811 22:18:39.634336 (+ 2568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:39.634342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:39.634417 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:39.634542 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:39.636835 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:39.637607 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:39.640195 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:39.640738 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:39.640876 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:39.645521 (+ 4645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:39.645527 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:39.645530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:39.645566 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:39.648422 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:39.649671 (+ 1249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:39.649676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:39.649681 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:39.649817 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:39.650305 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:39.650309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:39.650312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:39.650601 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:39.650767 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:39.651498 (+ 731us) server_negotiation.cc:299] Negotiation successful
0811 22:18:39.651735 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":258,"threads_started":1}
I20260811 22:18:39.653223 4194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:39.630592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:44257 (local address 127.29.229.2:35367)
0811 22:18:39.631560 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:39.631624 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:39.634068 (+ 2444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:39.634867 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:39.634876 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:39.635717 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:39.636608 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:39.636639 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:39.637792 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:39.637800 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:39.640002 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:39.640021 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:39.640987 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:39.640997 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:39.641343 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:39.642167 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:39.642204 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:39.645243 (+ 3039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:39.648627 (+ 3384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:39.648637 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:39.648760 (+ 123us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:39.649516 (+ 756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:39.649940 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:39.649945 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:39.649949 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:39.650168 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:39.650740 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:39.650749 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:39.651126 (+ 377us) client_negotiation.cc:769] Sending connection context
0811 22:18:39.651401 (+ 275us) client_negotiation.cc:240] Negotiation successful
0811 22:18:39.651975 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":338,"threads_started":1}
I20260811 22:18:39.654914 4187 heartbeater.cc:347] Connected to a master server at 127.29.229.62:44257
I20260811 22:18:39.655282 4187 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:39.656162 4187 heartbeater.cc:511] Master 127.29.229.62:44257 requested a full tablet report, sending...
I20260811 22:18:39.657858 3570 ts_manager.cc:194] Registered new tserver with Master: 9b730b4e966342ababa39ecbfbff14de (127.29.229.2:40967)
I20260811 22:18:39.658699 3570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:35367
WARNING: no policy specified for kudu/127.29.229.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
I20260811 22:18:39.692667 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:44257
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:40.067600 4197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:40.068019 4197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:40.068284 4197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:40.077783 4197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:40.077961 4197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:40.078104 4197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:18:40.089967 4197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:44257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:40.090642 3921 heartbeater.cc:503] Master 127.29.229.62:44257 was elected leader, sending a full tablet report...
I20260811 22:18:40.091683 4197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:40.093816 4197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:40.107576 4209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:40.107667 4210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:40.107692 4212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:40.109102 4197 server_base.cc:1034] running on GCE node
I20260811 22:18:40.109825 4197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:40.110849 4197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:40.112079 4197 hybrid_clock.cc:648] HybridClock initialized: now 1786486720112009 us; error 56 us; skew 500 ppm
I20260811 22:18:40.112438 4197 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:40 dist-test-slave-jq4t krb5kdc[3475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486720, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:40.119827 4197 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:18:40.121613 4197 webserver.cc:460] Webserver started at http://127.29.229.3:36989/ using document root <none> and password file <none>
I20260811 22:18:40.122419 4197 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:40.122581 4197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:40.122915 4197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:40.126329 4197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "b7ba470e18684826a51f534882d07e18"
format_stamp: "Formatted at 2026-08-11 22:18:40 on dist-test-slave-jq4t"
server_key: "477c0e73e8ca9870ec17e646e4788128"
server_key_iv: "e3a785f3fcea00b0f0beef3cdbfcc27b"
server_key_version: "encryptionkey@0"
I20260811 22:18:40.127327 4197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "b7ba470e18684826a51f534882d07e18"
format_stamp: "Formatted at 2026-08-11 22:18:40 on dist-test-slave-jq4t"
server_key: "477c0e73e8ca9870ec17e646e4788128"
server_key_iv: "e3a785f3fcea00b0f0beef3cdbfcc27b"
server_key_version: "encryptionkey@0"
I20260811 22:18:40.133391 4197 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 22:18:40.137346 4230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:40.139312 4197 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:18:40.139536 4197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "b7ba470e18684826a51f534882d07e18"
format_stamp: "Formatted at 2026-08-11 22:18:40 on dist-test-slave-jq4t"
server_key: "477c0e73e8ca9870ec17e646e4788128"
server_key_iv: "e3a785f3fcea00b0f0beef3cdbfcc27b"
server_key_version: "encryptionkey@0"
I20260811 22:18:40.139770 4197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:40.173980 4197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:40.179271 4197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:40.179628 4197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:40.181201 4197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:40.183475 4197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:40.183586 4197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:40.183750 4197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:40.183925 4197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:40.224020 4197 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:34385
I20260811 22:18:40.224061 4435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:34385 every 8 connection(s)
I20260811 22:18:40.225458 4197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:18:40.234447 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 4197
I20260811 22:18:40.234656 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:18:40.235169 30612 external_mini_cluster.cc:1428] Setting key 6d562459c2e0b25ac63dcc6cce52ab02
Aug 11 22:18:40 dist-test-slave-jq4t krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486720, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:40.249816 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:40.229112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:56381 (local address 127.29.229.62:44257)
0811 22:18:40.229878 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:40.229884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:40.230649 (+ 765us) server_negotiation.cc:407] Connection header received
0811 22:18:40.232638 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:40.232643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:40.232716 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:40.232838 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:40.234649 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:40.235572 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:40.237799 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:40.238252 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:40.238400 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:40.242323 (+ 3923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:40.242330 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:40.242334 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:40.242388 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:40.246358 (+ 3970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:40.247230 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:40.247235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:40.247237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:40.247390 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:40.247874 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:40.247878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:40.247880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:40.248184 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:40.248308 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:40.249146 (+ 838us) server_negotiation.cc:299] Negotiation successful
0811 22:18:40.249393 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":363,"threads_started":1}
I20260811 22:18:40.251139 4445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:40.229601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:44257 (local address 127.29.229.3:56381)
0811 22:18:40.230451 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:40.230507 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:40.232307 (+ 1800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:40.233087 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:40.233096 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:40.233630 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:40.234406 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:40.234436 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:40.235739 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:40.235743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:40.237645 (+ 1902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:40.237658 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:40.238508 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:40.238518 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:40.238796 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:40.239549 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:40.239573 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:40.241987 (+ 2414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:40.246540 (+ 4553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:40.246547 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:40.246561 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:40.247038 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:40.247542 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:40.247546 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:40.247548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:40.247756 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:40.248313 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:40.248319 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:40.248762 (+ 443us) client_negotiation.cc:769] Sending connection context
0811 22:18:40.249003 (+ 241us) client_negotiation.cc:240] Negotiation successful
0811 22:18:40.249578 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":389,"threads_started":1}
I20260811 22:18:40.252696 4440 heartbeater.cc:347] Connected to a master server at 127.29.229.62:44257
I20260811 22:18:40.253043 4440 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:40.253821 4440 heartbeater.cc:511] Master 127.29.229.62:44257 requested a full tablet report, sending...
I20260811 22:18:40.255873 3570 ts_manager.cc:194] Registered new tserver with Master: b7ba470e18684826a51f534882d07e18 (127.29.229.3:34385)
I20260811 22:18:40.257318 3570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:56381
I20260811 22:18:40.264621 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:18:40 dist-test-slave-jq4t krb5kdc[3475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486720, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 22:18:40 dist-test-slave-jq4t krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486720, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:40.332628 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:40.314254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41260 (local address 127.29.229.62:44257)
0811 22:18:40.314441 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:40.314446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:40.314845 (+ 399us) server_negotiation.cc:407] Connection header received
0811 22:18:40.315118 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:40.315122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:40.315220 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:40.315330 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:40.316960 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:40.317755 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:40.319423 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:40.319891 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:40.320003 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:40.325897 (+ 5894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:40.325906 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:40.325913 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:40.325981 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:40.329539 (+ 3558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:40.330419 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:40.330425 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:40.330428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:40.330584 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:40.331053 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:40.331058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:40.331062 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:40.331418 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:40.331698 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:40.332015 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 22:18:40.332264 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 22:18:40.333535 3570 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41260
I20260811 22:18:40.336012 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 3689
Aug 11 22:18:40 dist-test-slave-jq4t krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486720, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:18:40.355129 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:40.337886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40294 (local address 127.29.229.1:44489)
0811 22:18:40.338672 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:40.338679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:40.338705 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:18:40.338767 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:40.338774 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:40.339019 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:40.339200 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:40.340559 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:40.341637 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:40.342927 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:40.343777 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:40.343931 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:40.348653 (+ 4722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:40.348669 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:40.348685 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:40.348736 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:40.351918 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:40.352782 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:40.352790 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:40.352793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:40.352934 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:40.353344 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:40.353351 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:40.353358 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:40.353727 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:40.354059 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:40.354407 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 22:18:40.354700 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":427,"threads_started":1}
W20260811 22:18:40.356794 3913 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40294
W20260811 22:18:40.357992 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 3689: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:18:40.358268 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 3689
I20260811 22:18:40.391780 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 3939
Aug 11 22:18:40 dist-test-slave-jq4t krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486720, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.229.2@KRBTEST.COM
I20260811 22:18:40.412822 4464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:40.393214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50682 (local address 127.29.229.2:40967)
0811 22:18:40.393887 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:40.393894 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:40.393924 (+ 30us) server_negotiation.cc:407] Connection header received
0811 22:18:40.393995 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:40.394001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:40.394221 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:40.394389 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:40.395855 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:40.397042 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:40.398904 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:40.399551 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:40.399683 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:40.405386 (+ 5703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:40.405422 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:40.405445 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:40.405523 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:40.409256 (+ 3733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:40.410147 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:40.410156 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:40.410158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:40.410308 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:40.410725 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:40.410729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:40.410731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:40.411607 (+ 876us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:40.411869 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:40.412189 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 22:18:40.412419 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":354,"threads_started":1}
W20260811 22:18:40.414371 4179 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50682
W20260811 22:18:40.415560 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 3939: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:18:40.415709 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 3939
I20260811 22:18:40.441813 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 4197
Aug 11 22:18:40 dist-test-slave-jq4t krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486720, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.229.3@KRBTEST.COM
I20260811 22:18:40.463466 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:40.442909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52386 (local address 127.29.229.3:34385)
0811 22:18:40.443693 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:40.443701 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:40.443731 (+ 30us) server_negotiation.cc:407] Connection header received
0811 22:18:40.443799 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:40.443808 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:40.444044 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:40.444194 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:40.446717 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:40.447825 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:40.449499 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:40.450119 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:40.450343 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:40.455672 (+ 5329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:40.455686 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:40.455703 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:40.455743 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:40.459368 (+ 3625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:40.460159 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:40.460167 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:40.460169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:40.460314 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:40.461064 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:40.461179 (+ 115us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:40.461183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:40.461792 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:40.462150 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:40.462707 (+ 557us) server_negotiation.cc:299] Negotiation successful
0811 22:18:40.463010 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":421,"threads_started":1}
W20260811 22:18:40.464986 4430 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52386
W20260811 22:18:40.466461 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 4197: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:18:40.466645 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 4197
I20260811 22:18:40.496032 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 3502
I20260811 22:18:40.497781 3628 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:40.609829 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 3502
2026-08-11T22:18:40Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4379 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:18:40 dist-test-slave-jq4t krb5kdc[4474](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:18:40 dist-test-slave-jq4t krb5kdc[4474](info): set up 2 sockets
Aug 11 22:18:40 dist-test-slave-jq4t krb5kdc[4474](info): commencing operation
krb5kdc: starting...
W20260811 22:18:41.996110 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.003s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:18:42 dist-test-slave-jq4t krb5kdc[4474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:18:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:18:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:18:42.269098 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:45201
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:41029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:45201
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:42.624571 4498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:42.624883 4498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:42.624985 4498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:42.632539 4498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:42.632666 4498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:42.632714 4498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:42.632766 4498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:42.632812 4498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:18:42.642724 4498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:41029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:45201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:45201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:42.644042 4498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:42.645411 4498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:42.657406 4512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:42.657320 4515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:42.657456 4518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:42.658407 4498 server_base.cc:1034] running on GCE node
I20260811 22:18:42.659233 4498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:42.660691 4498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:42.661936 4498 hybrid_clock.cc:648] HybridClock initialized: now 1786486722661885 us; error 37 us; skew 500 ppm
I20260811 22:18:42.662367 4498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:42 dist-test-slave-jq4t krb5kdc[4474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486722, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:42.665753 4498 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:18:42.667382 4498 webserver.cc:460] Webserver started at http://127.29.229.62:36929/ using document root <none> and password file <none>
I20260811 22:18:42.668114 4498 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:42.668267 4498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:42.668671 4498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:42.671720 4498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "72488aca416e4d8fafd0bc29f5c7a61c"
format_stamp: "Formatted at 2026-08-11 22:18:42 on dist-test-slave-jq4t"
server_key: "3b817ca1c77fc82da08d145f8ee0a4c4"
server_key_iv: "b956de20896dd540e21b060d8b1dc5d6"
server_key_version: "encryptionkey@0"
I20260811 22:18:42.672616 4498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "72488aca416e4d8fafd0bc29f5c7a61c"
format_stamp: "Formatted at 2026-08-11 22:18:42 on dist-test-slave-jq4t"
server_key: "3b817ca1c77fc82da08d145f8ee0a4c4"
server_key_iv: "b956de20896dd540e21b060d8b1dc5d6"
server_key_version: "encryptionkey@0"
I20260811 22:18:42.678171 4498 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 22:18:42.681995 4532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:42.683564 4498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:18:42.683784 4498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "72488aca416e4d8fafd0bc29f5c7a61c"
format_stamp: "Formatted at 2026-08-11 22:18:42 on dist-test-slave-jq4t"
server_key: "3b817ca1c77fc82da08d145f8ee0a4c4"
server_key_iv: "b956de20896dd540e21b060d8b1dc5d6"
server_key_version: "encryptionkey@0"
I20260811 22:18:42.684021 4498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:42.712241 4498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:42.716894 4498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:42.717216 4498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:42.738268 4498 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:45201
I20260811 22:18:42.738291 4619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:45201 every 8 connection(s)
I20260811 22:18:42.739722 4498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:42.743489 4621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:42.747259 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 4498
I20260811 22:18:42.747541 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:18:42.748212 30612 external_mini_cluster.cc:1428] Setting key 11ab568bed55e2078aa73e75a4ca8eee
I20260811 22:18:42.755522 4621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c: Bootstrap starting.
I20260811 22:18:42.759449 4621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:42.760754 4621 log.cc:824] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:18:42 dist-test-slave-jq4t krb5kdc[4474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:42.764837 4621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c: No bootstrap required, opened a new log
I20260811 22:18:42.773470 4621 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72488aca416e4d8fafd0bc29f5c7a61c" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45201 } }
I20260811 22:18:42.773887 4624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:42.750451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39124 (local address 127.29.229.62:45201)
0811 22:18:42.751306 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:42.751318 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:42.751360 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:18:42.753638 (+ 2278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:42.753663 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:42.754057 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:42.754385 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:42.756397 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:42.757838 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:42.759797 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:42.760585 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:42.760744 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:42.765670 (+ 4926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:42.765678 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:42.765693 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:42.765730 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:42.769103 (+ 3373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:42.770135 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:42.770156 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:42.770164 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:42.770368 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:42.770861 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:42.770867 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:42.770870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:42.771470 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:42.771823 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:42.772312 (+ 489us) server_negotiation.cc:299] Negotiation successful
0811 22:18:42.772802 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":385,"threads_started":1}
I20260811 22:18:42.773918 4621 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:42.774047 4621 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72488aca416e4d8fafd0bc29f5c7a61c, State: Initialized, Role: FOLLOWER
I20260811 22:18:42.774647 4621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72488aca416e4d8fafd0bc29f5c7a61c" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45201 } }
I20260811 22:18:42.774928 4621 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:18:42.775110 4621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:18:42.775313 4621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:42.778407 4621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72488aca416e4d8fafd0bc29f5c7a61c" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45201 } }
I20260811 22:18:42.779019 4621 leader_election.cc:302] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72488aca416e4d8fafd0bc29f5c7a61c; no voters:
I20260811 22:18:42.779991 4621 leader_election.cc:288] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:18:42.780112 4627 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:42.781069 4627 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c [term 1 LEADER]: Becoming Leader. State: Replica: 72488aca416e4d8fafd0bc29f5c7a61c, State: Running, Role: LEADER
I20260811 22:18:42.781638 4627 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72488aca416e4d8fafd0bc29f5c7a61c" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45201 } }
I20260811 22:18:42.783375 4621 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:42.785467 4629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72488aca416e4d8fafd0bc29f5c7a61c. Latest consensus state: current_term: 1 leader_uuid: "72488aca416e4d8fafd0bc29f5c7a61c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72488aca416e4d8fafd0bc29f5c7a61c" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45201 } } }
I20260811 22:18:42.785627 4628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72488aca416e4d8fafd0bc29f5c7a61c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72488aca416e4d8fafd0bc29f5c7a61c" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45201 } } }
I20260811 22:18:42.786028 4628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:42.786032 4629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:42.787767 4638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:42.794507 4638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:42.804706 4638 catalog_manager.cc:1347] Generated new cluster ID: e88c5c4fa67f4d0f8ea880b0966a9680
I20260811 22:18:42.804945 4638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:42.815260 4638 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:18:42.817028 4638 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:42.832643 4638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72488aca416e4d8fafd0bc29f5c7a61c: Generated new TSK 0
I20260811 22:18:42.833604 4638 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
I20260811 22:18:42.904734 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:45201
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:41029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:43.262470 4660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:43.262964 4660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:43.263178 4660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:43.270685 4660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:43.270874 4660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:43.271052 4660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:18:43.281572 4660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:41029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:45201
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:43.283003 4660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:43.284535 4660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:43.298943 4669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:43.299131 4668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:43.298966 4673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:43.300038 4660 server_base.cc:1034] running on GCE node
I20260811 22:18:43.300827 4660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:43.301970 4660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:43.303239 4660 hybrid_clock.cc:648] HybridClock initialized: now 1786486723303161 us; error 95 us; skew 500 ppm
I20260811 22:18:43.303612 4660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:43 dist-test-slave-jq4t krb5kdc[4474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486723, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:43.307227 4660 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:18:43.309254 4660 webserver.cc:460] Webserver started at http://127.29.229.1:44757/ using document root <none> and password file <none>
I20260811 22:18:43.310031 4660 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:43.310175 4660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:43.310541 4660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:43.314121 4660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "31daa3f09a1e4b18beb3820552f1042b"
format_stamp: "Formatted at 2026-08-11 22:18:43 on dist-test-slave-jq4t"
server_key: "a093f3e8a5c1cd31acec4afe3cbb5471"
server_key_iv: "4add7f5a8b442edea13b38f2e51eb9e7"
server_key_version: "encryptionkey@0"
I20260811 22:18:43.315271 4660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "31daa3f09a1e4b18beb3820552f1042b"
format_stamp: "Formatted at 2026-08-11 22:18:43 on dist-test-slave-jq4t"
server_key: "a093f3e8a5c1cd31acec4afe3cbb5471"
server_key_iv: "4add7f5a8b442edea13b38f2e51eb9e7"
server_key_version: "encryptionkey@0"
I20260811 22:18:43.321133 4660 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 22:18:43.325050 4694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:43.326452 4660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:18:43.326646 4660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "31daa3f09a1e4b18beb3820552f1042b"
format_stamp: "Formatted at 2026-08-11 22:18:43 on dist-test-slave-jq4t"
server_key: "a093f3e8a5c1cd31acec4afe3cbb5471"
server_key_iv: "4add7f5a8b442edea13b38f2e51eb9e7"
server_key_version: "encryptionkey@0"
I20260811 22:18:43.326861 4660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:43.349685 4660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:43.354277 4660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:43.354612 4660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:43.355671 4660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:43.357156 4660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:43.357230 4660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:43.357404 4660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:43.357465 4660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:43.398749 4660 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:46601
I20260811 22:18:43.398800 4918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:46601 every 8 connection(s)
I20260811 22:18:43.400101 4660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:18:43.404645 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 4660
I20260811 22:18:43.404903 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:18:43.405792 30612 external_mini_cluster.cc:1428] Setting key 8ab9d9c28febe71b86c660d416917e5b
Aug 11 22:18:43 dist-test-slave-jq4t krb5kdc[4474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486723, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:43.426672 4925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:43.402824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:34601 (local address 127.29.229.62:45201)
0811 22:18:43.403325 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:43.403330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:43.404357 (+ 1027us) server_negotiation.cc:407] Connection header received
0811 22:18:43.407114 (+ 2757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:43.407119 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:43.407212 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:43.407335 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:43.409593 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:43.410376 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:43.413099 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:43.413542 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:43.413656 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:43.418675 (+ 5019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:43.418681 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:43.418684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:43.418722 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:43.422303 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:43.423401 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:43.423406 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:43.423408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:43.423574 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:43.424156 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:43.424159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:43.424161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:43.424452 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:43.424653 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:43.425909 (+ 1256us) server_negotiation.cc:299] Negotiation successful
0811 22:18:43.426275 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":289,"threads_started":1}
I20260811 22:18:43.427805 4929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:43.403307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45201 (local address 127.29.229.1:34601)
0811 22:18:43.404119 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:43.404181 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:43.406754 (+ 2573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:43.407583 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:43.407593 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:43.408419 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:43.409378 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:43.409410 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:43.410575 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:43.410583 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:43.412896 (+ 2313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:43.412917 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:43.413813 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:43.413823 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:43.414191 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:43.415076 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:43.415120 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:43.418361 (+ 3241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:43.422482 (+ 4121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:43.422495 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:43.422519 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:43.423227 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:43.423713 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:43.423721 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:43.423728 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:43.423998 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:43.424658 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:43.424669 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:43.425194 (+ 525us) client_negotiation.cc:769] Sending connection context
0811 22:18:43.425522 (+ 328us) client_negotiation.cc:240] Negotiation successful
0811 22:18:43.426237 (+ 715us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":287,"threads_started":1}
I20260811 22:18:43.429822 4922 heartbeater.cc:347] Connected to a master server at 127.29.229.62:45201
I20260811 22:18:43.430205 4922 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:43.431223 4922 heartbeater.cc:511] Master 127.29.229.62:45201 requested a full tablet report, sending...
I20260811 22:18:43.434281 4560 ts_manager.cc:194] Registered new tserver with Master: 31daa3f09a1e4b18beb3820552f1042b (127.29.229.1:46601)
I20260811 22:18:43.437129 4560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:34601
WARNING: no policy specified for kudu/127.29.229.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
I20260811 22:18:43.477845 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:45201
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:41029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:43.848114 4937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:43.848413 4937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:43.848608 4937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:43.856357 4937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:43.856539 4937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:43.856760 4937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:18:43.866995 4937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:41029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:45201
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:43.868357 4937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:43.869743 4937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:43.883881 4942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:43.884359 4944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:43.884739 4948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:43.885990 4937 server_base.cc:1034] running on GCE node
I20260811 22:18:43.886684 4937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:43.888077 4937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:43.889317 4937 hybrid_clock.cc:648] HybridClock initialized: now 1786486723889281 us; error 55 us; skew 500 ppm
I20260811 22:18:43.889679 4937 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:43 dist-test-slave-jq4t krb5kdc[4474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486723, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:43.893464 4937 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:18:43.895130 4937 webserver.cc:460] Webserver started at http://127.29.229.2:43953/ using document root <none> and password file <none>
I20260811 22:18:43.895864 4937 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:43.896025 4937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:43.896366 4937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:43.899466 4937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "b10caa194cff463e830cd254e72fd466"
format_stamp: "Formatted at 2026-08-11 22:18:43 on dist-test-slave-jq4t"
server_key: "7dce99d579e9f957602f7244796a0440"
server_key_iv: "bd44f5036263b0689014fde7795c2dd2"
server_key_version: "encryptionkey@0"
I20260811 22:18:43.900313 4937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "b10caa194cff463e830cd254e72fd466"
format_stamp: "Formatted at 2026-08-11 22:18:43 on dist-test-slave-jq4t"
server_key: "7dce99d579e9f957602f7244796a0440"
server_key_iv: "bd44f5036263b0689014fde7795c2dd2"
server_key_version: "encryptionkey@0"
I20260811 22:18:43.905964 4937 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 22:18:43.909662 4966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:43.911125 4937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 22:18:43.911319 4937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "b10caa194cff463e830cd254e72fd466"
format_stamp: "Formatted at 2026-08-11 22:18:43 on dist-test-slave-jq4t"
server_key: "7dce99d579e9f957602f7244796a0440"
server_key_iv: "bd44f5036263b0689014fde7795c2dd2"
server_key_version: "encryptionkey@0"
I20260811 22:18:43.911540 4937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:43.926905 4937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:43.932579 4937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:43.932893 4937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:43.934006 4937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:43.935451 4937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:43.935570 4937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:43.935741 4937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:43.935820 4937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:43.972774 4937 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:36673
I20260811 22:18:43.972812 5204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:36673 every 8 connection(s)
I20260811 22:18:43.974097 4937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:18:43.977290 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 4937
I20260811 22:18:43.977540 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:18:43.978188 30612 external_mini_cluster.cc:1428] Setting key 57e4b3ff53c3d37d4a05586e53402e6a
Aug 11 22:18:43 dist-test-slave-jq4t krb5kdc[4474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486723, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:44.002079 5209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:43.977681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:55063 (local address 127.29.229.62:45201)
0811 22:18:43.978373 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:43.978381 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:43.979988 (+ 1607us) server_negotiation.cc:407] Connection header received
0811 22:18:43.982757 (+ 2769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:43.982762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:43.982937 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:43.983056 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:43.986144 (+ 3088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:43.986972 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:43.989700 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:43.990158 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:43.990266 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:43.995149 (+ 4883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:43.995156 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:43.995247 (+ 91us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:43.995302 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:43.998232 (+ 2930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:43.999331 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:43.999335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:43.999337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:43.999469 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:44.000063 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:44.000067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:44.000070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:44.000386 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:44.000599 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:44.001546 (+ 947us) server_negotiation.cc:299] Negotiation successful
0811 22:18:44.001743 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":342,"threads_started":1}
I20260811 22:18:44.003693 5210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:43.978660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45201 (local address 127.29.229.2:55063)
0811 22:18:43.979772 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:43.979826 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:43.982410 (+ 2584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:43.983550 (+ 1140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:43.983569 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:43.984630 (+ 1061us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:43.985869 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:43.985922 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:43.987164 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:43.987172 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:43.989536 (+ 2364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:43.989556 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:43.990420 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:43.990430 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:43.990787 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:43.991669 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:43.991706 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:43.994694 (+ 2988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:43.998414 (+ 3720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:43.998426 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:43.998448 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:43.999162 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:43.999593 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:43.999601 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:43.999607 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:43.999889 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:44.000565 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:44.000576 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:44.001107 (+ 531us) client_negotiation.cc:769] Sending connection context
0811 22:18:44.001427 (+ 320us) client_negotiation.cc:240] Negotiation successful
0811 22:18:44.002125 (+ 698us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":475,"threads_started":1}
I20260811 22:18:44.005359 5205 heartbeater.cc:347] Connected to a master server at 127.29.229.62:45201
I20260811 22:18:44.005748 5205 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:44.006773 5205 heartbeater.cc:511] Master 127.29.229.62:45201 requested a full tablet report, sending...
I20260811 22:18:44.008616 4560 ts_manager.cc:194] Registered new tserver with Master: b10caa194cff463e830cd254e72fd466 (127.29.229.2:36673)
WARNING: no policy specified for kudu/127.29.229.3@KRBTEST.COM; defaulting to no policy
I20260811 22:18:44.009552 4560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:55063
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
I20260811 22:18:44.042264 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:45201
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:41029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:44.406581 5220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:44.406922 5220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:44.407104 5220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:44.414817 5220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:44.414997 5220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:44.415215 5220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:18:44.425865 5220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:41029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:45201
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:44.427316 5220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:44.428774 5220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:18:44.442008 4922 heartbeater.cc:503] Master 127.29.229.62:45201 was elected leader, sending a full tablet report...
W20260811 22:18:44.442304 5232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:44.442396 5239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:44.442800 5234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:44.443647 5220 server_base.cc:1034] running on GCE node
I20260811 22:18:44.444417 5220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:44.445443 5220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:44.446679 5220 hybrid_clock.cc:648] HybridClock initialized: now 1786486724446629 us; error 40 us; skew 500 ppm
I20260811 22:18:44.447062 5220 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:44 dist-test-slave-jq4t krb5kdc[4474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486724, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:44.450533 5220 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:18:44.452248 5220 webserver.cc:460] Webserver started at http://127.29.229.3:45537/ using document root <none> and password file <none>
I20260811 22:18:44.453078 5220 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:44.453243 5220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:44.453591 5220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:44.456988 5220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "88754c73340e432493fbbf6f78ccfd1d"
format_stamp: "Formatted at 2026-08-11 22:18:44 on dist-test-slave-jq4t"
server_key: "c73abdb197fcd27848f3d2c1f8dedb4d"
server_key_iv: "3589a13ee64c7c24af56408175cd8000"
server_key_version: "encryptionkey@0"
I20260811 22:18:44.457819 5220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "88754c73340e432493fbbf6f78ccfd1d"
format_stamp: "Formatted at 2026-08-11 22:18:44 on dist-test-slave-jq4t"
server_key: "c73abdb197fcd27848f3d2c1f8dedb4d"
server_key_iv: "3589a13ee64c7c24af56408175cd8000"
server_key_version: "encryptionkey@0"
I20260811 22:18:44.463305 5220 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:18:44.467074 5256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:44.468978 5220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:18:44.469173 5220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "88754c73340e432493fbbf6f78ccfd1d"
format_stamp: "Formatted at 2026-08-11 22:18:44 on dist-test-slave-jq4t"
server_key: "c73abdb197fcd27848f3d2c1f8dedb4d"
server_key_iv: "3589a13ee64c7c24af56408175cd8000"
server_key_version: "encryptionkey@0"
I20260811 22:18:44.469417 5220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:44.501590 5220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:44.506332 5220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:44.506660 5220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:44.507710 5220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:44.509202 5220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:44.509308 5220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:44.509461 5220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:44.509546 5220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:44.548846 5220 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:42737
I20260811 22:18:44.549091 5476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:42737 every 8 connection(s)
I20260811 22:18:44.550366 5220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:18:44.551394 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 5220
I20260811 22:18:44.551638 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:18:44.552230 30612 external_mini_cluster.cc:1428] Setting key ed10979bbdd6f85262d9f8ebd2f4f167
Aug 11 22:18:44 dist-test-slave-jq4t krb5kdc[4474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486724, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:44.574512 5481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:44.554223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:37305 (local address 127.29.229.62:45201)
0811 22:18:44.554827 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:44.554832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:44.556095 (+ 1263us) server_negotiation.cc:407] Connection header received
0811 22:18:44.558193 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:44.558197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:44.558271 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:44.558374 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:44.560009 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:44.561038 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:44.563233 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:44.563652 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:44.563760 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:44.567825 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:44.567832 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:44.567836 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:44.567869 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:44.571135 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:44.571965 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:44.571973 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:44.571975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:44.572110 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:44.572618 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:44.572621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:44.572623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:44.572935 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:44.573117 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:44.573876 (+ 759us) server_negotiation.cc:299] Negotiation successful
0811 22:18:44.574123 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":298,"threads_started":1}
I20260811 22:18:44.575594 5482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:44.555007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45201 (local address 127.29.229.3:37305)
0811 22:18:44.555922 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:44.555972 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:44.557896 (+ 1924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:44.558606 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:44.558615 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:44.559168 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:44.559786 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:44.559804 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:44.561363 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:44.561369 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:44.563054 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:44.563073 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:44.563884 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:44.563890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:44.564128 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:44.564881 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:44.564906 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:44.567577 (+ 2671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:44.571298 (+ 3721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:44.571307 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:44.571320 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:44.571797 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:44.572226 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:44.572230 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:44.572232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:44.572434 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:44.573084 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:44.573093 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:44.573477 (+ 384us) client_negotiation.cc:769] Sending connection context
0811 22:18:44.573770 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 22:18:44.574341 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":416,"threads_started":1}
I20260811 22:18:44.577405 5478 heartbeater.cc:347] Connected to a master server at 127.29.229.62:45201
I20260811 22:18:44.577715 5478 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:44.578469 5478 heartbeater.cc:511] Master 127.29.229.62:45201 requested a full tablet report, sending...
I20260811 22:18:44.580145 4560 ts_manager.cc:194] Registered new tserver with Master: 88754c73340e432493fbbf6f78ccfd1d (127.29.229.3:42737)
I20260811 22:18:44.581205 4560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:37305
I20260811 22:18:44.582597 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:18:44 dist-test-slave-jq4t krb5kdc[4474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:18:44 dist-test-slave-jq4t krb5kdc[4474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:18:44.654012 5491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:44.635249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35600 (local address 127.29.229.1:46601)
0811 22:18:44.636130 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:44.636138 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:44.636164 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:18:44.636229 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:44.636236 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:44.636453 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:44.636684 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:44.638196 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:44.639266 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:44.640547 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:44.641166 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:44.641318 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:44.646583 (+ 5265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:44.646599 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:44.646615 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:44.646656 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:44.650699 (+ 4043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:44.651493 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:44.651501 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:44.651504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:44.651641 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:44.652089 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:44.652093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:44.652095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:44.652607 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:44.653016 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:44.653301 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 22:18:44.653625 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":382,"threads_started":1}
Aug 11 22:18:44 dist-test-slave-jq4t krb5kdc[4474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:44.673904 5481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:44.659324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39128 (local address 127.29.229.62:45201)
0811 22:18:44.659566 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:44.659573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:44.659818 (+ 245us) server_negotiation.cc:407] Connection header received
0811 22:18:44.659933 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:44.659937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:44.660015 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:44.660103 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:44.661506 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:44.662262 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:44.663586 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:44.664049 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:44.664160 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:44.668336 (+ 4176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:44.668341 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:44.668345 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:44.668391 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:44.671210 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:44.671879 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:44.671883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:44.671885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:44.672023 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:44.672372 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:44.672375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:44.672376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:44.672788 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:44.673000 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:44.673360 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 22:18:44.673540 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 22:18:44.674548 4560 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39128
I20260811 22:18:44.675655 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 4660
I20260811 22:18:44.689620 5491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:44.676771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35604 (local address 127.29.229.1:46601)
0811 22:18:44.677046 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:44.677053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:44.677213 (+ 160us) server_negotiation.cc:407] Connection header received
0811 22:18:44.677426 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:44.677431 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:44.677531 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:44.677646 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:44.678897 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:44.679614 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:44.680921 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:44.681367 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:44.681514 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:44.683791 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:44.683796 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:44.683799 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:44.683829 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:44.686625 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:44.687326 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:44.687335 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:44.687339 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:44.687471 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:44.687987 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:44.687995 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:44.687998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:44.688352 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:44.688648 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:44.689017 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 22:18:44.689263 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260811 22:18:44.690294 4906 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:44.785245 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 4660
I20260811 22:18:44.813823 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 4937
Aug 11 22:18:44 dist-test-slave-jq4t krb5kdc[4474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.2@KRBTEST.COM
I20260811 22:18:44.836658 5501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:44.814907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39150 (local address 127.29.229.2:36673)
0811 22:18:44.815612 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:44.815620 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:44.815647 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:18:44.815711 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:44.815717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:44.815962 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:44.816101 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:44.817957 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:44.819022 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:44.820316 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:44.821250 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:44.821374 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:44.830095 (+ 8721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:44.830114 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:44.830131 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:44.830173 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:44.833301 (+ 3128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:44.834076 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:44.834084 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:44.834087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:44.834229 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:44.834746 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:44.834750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:44.834752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:44.835135 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:44.835394 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:44.835909 (+ 515us) server_negotiation.cc:299] Negotiation successful
0811 22:18:44.836145 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":392,"threads_started":1}
I20260811 22:18:44.838246 5196 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:44.923491 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 4937
I20260811 22:18:44.952346 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 5220
Aug 11 22:18:44 dist-test-slave-jq4t krb5kdc[4474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.3@KRBTEST.COM
I20260811 22:18:44.982617 5505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:44.954295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35702 (local address 127.29.229.3:42737)
0811 22:18:44.955235 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:44.955247 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:44.955293 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:18:44.955392 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:44.955405 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:44.955737 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:44.956230 (+ 493us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:44.958548 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:44.960500 (+ 1952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:44.963112 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:44.964546 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:44.964756 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:44.970553 (+ 5797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:44.970572 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:44.970614 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:44.970676 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:44.977632 (+ 6956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:44.978541 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:44.978553 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:44.978557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:44.978710 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:44.979124 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:44.979129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:44.979133 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:44.979544 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:44.979863 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:44.981574 (+ 1711us) server_negotiation.cc:299] Negotiation successful
0811 22:18:44.982025 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":434,"threads_started":1}
I20260811 22:18:44.985306 5469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:45.091042 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 5220
I20260811 22:18:45.123306 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 4498
I20260811 22:18:45.124727 4613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:45.232818 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 4498
2026-08-11T22:18:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4628 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:18:45 dist-test-slave-jq4t krb5kdc[5517](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:18:45 dist-test-slave-jq4t krb5kdc[5517](info): set up 2 sockets
Aug 11 22:18:45 dist-test-slave-jq4t krb5kdc[5517](info): commencing operation
krb5kdc: starting...
W20260811 22:18:46.628652 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.335s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:18:46 dist-test-slave-jq4t krb5kdc[5517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:18:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:18:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:18:46.912312 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:43515
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:44739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:43515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:47.291841 5543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:47.292172 5543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:47.292276 5543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:47.299898 5543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:47.300036 5543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:47.300137 5543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:47.300256 5543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:47.300328 5543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:18:47.310839 5543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:44739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:43515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:43515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:47.312314 5543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:47.313896 5543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:47.327339 5554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:47.327448 5557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:47.327417 5552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:47.328550 5543 server_base.cc:1034] running on GCE node
I20260811 22:18:47.329490 5543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:47.331032 5543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:47.332309 5543 hybrid_clock.cc:648] HybridClock initialized: now 1786486727332265 us; error 56 us; skew 500 ppm
I20260811 22:18:47.332765 5543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:47 dist-test-slave-jq4t krb5kdc[5517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486727, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:47.337172 5543 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:18:47.338400 5543 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:47.338495 5543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:47.338883 5543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:47.342124 5543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "cf72ac26cc5041d1af24b7e939c2450c"
format_stamp: "Formatted at 2026-08-11 22:18:47 on dist-test-slave-jq4t"
server_key: "b56ad979115992692e1d1546f1100790"
server_key_iv: "e319c81f14eea99d5a01d805dfe466ae"
server_key_version: "encryptionkey@0"
I20260811 22:18:47.342993 5543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "cf72ac26cc5041d1af24b7e939c2450c"
format_stamp: "Formatted at 2026-08-11 22:18:47 on dist-test-slave-jq4t"
server_key: "b56ad979115992692e1d1546f1100790"
server_key_iv: "e319c81f14eea99d5a01d805dfe466ae"
server_key_version: "encryptionkey@0"
I20260811 22:18:47.348872 5543 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 22:18:47.353045 5573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:47.354739 5543 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:18:47.354964 5543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "cf72ac26cc5041d1af24b7e939c2450c"
format_stamp: "Formatted at 2026-08-11 22:18:47 on dist-test-slave-jq4t"
server_key: "b56ad979115992692e1d1546f1100790"
server_key_iv: "e319c81f14eea99d5a01d805dfe466ae"
server_key_version: "encryptionkey@0"
I20260811 22:18:47.355302 5543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:47.371366 5543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:47.375699 5543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:47.375948 5543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:47.398378 5543 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:43515
I20260811 22:18:47.398399 5663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:43515 every 8 connection(s)
I20260811 22:18:47.399366 5543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:47.403062 5666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:47.403577 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 5543
I20260811 22:18:47.403812 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:18:47.404465 30612 external_mini_cluster.cc:1428] Setting key 9f40f3533b73b84304373f6cdb3a2dba
I20260811 22:18:47.418632 5666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c: Bootstrap starting.
Aug 11 22:18:47 dist-test-slave-jq4t krb5kdc[5517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:47.422251 5666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:47.423703 5666 log.cc:824] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:47.427249 5666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c: No bootstrap required, opened a new log
I20260811 22:18:47.430730 5673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:47.406613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56888 (local address 127.29.229.62:43515)
0811 22:18:47.407507 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:47.407522 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:47.407573 (+ 51us) server_negotiation.cc:407] Connection header received
0811 22:18:47.410384 (+ 2811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:47.410416 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:47.410972 (+ 556us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:47.411453 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:47.413007 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:47.415099 (+ 2092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:47.416447 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:47.417575 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:47.417780 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:47.421942 (+ 4162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:47.421956 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:47.421981 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:47.422038 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:47.425973 (+ 3935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:47.426690 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:47.426698 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:47.426705 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:47.426922 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:47.427294 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:47.427301 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:47.427305 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:47.428099 (+ 794us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:47.428422 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:47.428846 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 22:18:47.429305 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":364,"threads_started":1}
I20260811 22:18:47.436852 5666 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf72ac26cc5041d1af24b7e939c2450c" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43515 } }
I20260811 22:18:47.437206 5666 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:47.437325 5666 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf72ac26cc5041d1af24b7e939c2450c, State: Initialized, Role: FOLLOWER
I20260811 22:18:47.437824 5666 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf72ac26cc5041d1af24b7e939c2450c" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43515 } }
I20260811 22:18:47.438032 5666 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:18:47.438257 5666 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:18:47.438454 5666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:47.441890 5666 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf72ac26cc5041d1af24b7e939c2450c" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43515 } }
I20260811 22:18:47.442435 5666 leader_election.cc:302] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf72ac26cc5041d1af24b7e939c2450c; no voters:
I20260811 22:18:47.443140 5666 leader_election.cc:288] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:18:47.443264 5675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:47.444711 5675 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c [term 1 LEADER]: Becoming Leader. State: Replica: cf72ac26cc5041d1af24b7e939c2450c, State: Running, Role: LEADER
I20260811 22:18:47.445323 5675 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf72ac26cc5041d1af24b7e939c2450c" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43515 } }
I20260811 22:18:47.445827 5666 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:47.448441 5681 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf72ac26cc5041d1af24b7e939c2450c. Latest consensus state: current_term: 1 leader_uuid: "cf72ac26cc5041d1af24b7e939c2450c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf72ac26cc5041d1af24b7e939c2450c" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43515 } } }
I20260811 22:18:47.448851 5681 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:47.449476 5677 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf72ac26cc5041d1af24b7e939c2450c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf72ac26cc5041d1af24b7e939c2450c" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43515 } } }
I20260811 22:18:47.449841 5677 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:47.450872 5693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:47.457320 5693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:47.466816 5693 catalog_manager.cc:1347] Generated new cluster ID: 692160c5f66e4beb9d3744552f20dead
I20260811 22:18:47.466981 5693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:47.511749 5693 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:18:47.513540 5693 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:47.523449 5693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c: Generated new TSK 0
I20260811 22:18:47.524827 5693 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
I20260811 22:18:47.592650 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43515
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:44739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:47.964102 5713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:47.964453 5713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:47.964686 5713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:47.972091 5713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:47.972252 5713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:47.972445 5713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:18:47.983106 5713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:44739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:47.985313 5713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:47.987398 5713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:48.001579 5725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:48.001688 5721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:48.001746 5730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:48.002741 5713 server_base.cc:1034] running on GCE node
I20260811 22:18:48.003384 5713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:48.004398 5713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:48.005666 5713 hybrid_clock.cc:648] HybridClock initialized: now 1786486728005630 us; error 38 us; skew 500 ppm
I20260811 22:18:48.006048 5713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:48 dist-test-slave-jq4t krb5kdc[5517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486728, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:48.009479 5713 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:18:48.010701 5713 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:48.010809 5713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:48.011096 5713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:48.014660 5713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "2eb73c6a6cc542648d67680df8471efe"
format_stamp: "Formatted at 2026-08-11 22:18:48 on dist-test-slave-jq4t"
server_key: "5d4e7a4293a0e59fb0b36fda8ce59bb9"
server_key_iv: "e3033ae189fcc5181ef34d2ee96f99c6"
server_key_version: "encryptionkey@0"
I20260811 22:18:48.015486 5713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "2eb73c6a6cc542648d67680df8471efe"
format_stamp: "Formatted at 2026-08-11 22:18:48 on dist-test-slave-jq4t"
server_key: "5d4e7a4293a0e59fb0b36fda8ce59bb9"
server_key_iv: "e3033ae189fcc5181ef34d2ee96f99c6"
server_key_version: "encryptionkey@0"
I20260811 22:18:48.021593 5713 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 22:18:48.026523 5740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:48.028527 5713 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 22:18:48.028719 5713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "2eb73c6a6cc542648d67680df8471efe"
format_stamp: "Formatted at 2026-08-11 22:18:48 on dist-test-slave-jq4t"
server_key: "5d4e7a4293a0e59fb0b36fda8ce59bb9"
server_key_iv: "e3033ae189fcc5181ef34d2ee96f99c6"
server_key_version: "encryptionkey@0"
I20260811 22:18:48.028954 5713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:48.048961 5713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:48.053864 5713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:48.054114 5713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:48.055091 5713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:48.056540 5713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:48.056631 5713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:48.056725 5713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:48.056784 5713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:48.093291 5713 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:43947
I20260811 22:18:48.093348 5938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:43947 every 8 connection(s)
I20260811 22:18:48.094215 5713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:18:48.102120 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 5713
I20260811 22:18:48.102357 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:18:48.103013 30612 external_mini_cluster.cc:1428] Setting key 77645068b98acfb59a9945f0a6cfb193
Aug 11 22:18:48 dist-test-slave-jq4t krb5kdc[5517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486728, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:48.116518 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:48.096792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:43907 (local address 127.29.229.62:43515)
0811 22:18:48.097361 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:48.097389 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:48.098057 (+ 668us) server_negotiation.cc:407] Connection header received
0811 22:18:48.100085 (+ 2028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:48.100091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:48.100195 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:48.100333 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:48.102012 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:48.102978 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:48.104864 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:48.105328 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:48.105493 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:48.109559 (+ 4066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:48.109565 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:48.109569 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:48.109607 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:48.112675 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:48.113926 (+ 1251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:48.113936 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:48.113938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:48.114082 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:48.114562 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:48.114572 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:48.114574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:48.114871 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:48.115050 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:48.115829 (+ 779us) server_negotiation.cc:299] Negotiation successful
0811 22:18:48.116096 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":295,"threads_started":1}
I20260811 22:18:48.117586 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:48.097116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43515 (local address 127.29.229.1:43907)
0811 22:18:48.097886 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:48.097928 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:48.099779 (+ 1851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:48.100628 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:48.100637 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:48.101207 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:48.101805 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:48.101824 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:48.103199 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:48.103204 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:48.104690 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:48.104705 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:48.105602 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:48.105613 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:48.105875 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:48.106651 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:48.106675 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:48.109282 (+ 2607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:48.112838 (+ 3556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:48.112850 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:48.112869 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:48.113758 (+ 889us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:48.114205 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:48.114210 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:48.114214 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:48.114421 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:48.115020 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:48.115029 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:48.115412 (+ 383us) client_negotiation.cc:769] Sending connection context
0811 22:18:48.115708 (+ 296us) client_negotiation.cc:240] Negotiation successful
0811 22:18:48.116306 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":338,"threads_started":1}
I20260811 22:18:48.119953 5939 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43515
I20260811 22:18:48.120362 5939 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:48.121402 5939 heartbeater.cc:511] Master 127.29.229.62:43515 requested a full tablet report, sending...
I20260811 22:18:48.124184 5606 ts_manager.cc:194] Registered new tserver with Master: 2eb73c6a6cc542648d67680df8471efe (127.29.229.1:43947)
I20260811 22:18:48.125909 5606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:43907
WARNING: no policy specified for kudu/127.29.229.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
I20260811 22:18:48.168054 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43515
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:44739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:48.529134 5946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:48.529443 5946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:48.529547 5946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:48.537851 5946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:48.538060 5946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:48.538213 5946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:18:48.549233 5946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:44739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:48.550637 5946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:48.552093 5946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:48.566309 5955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:48.566238 5965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:48.566285 5961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:48.567714 5946 server_base.cc:1034] running on GCE node
I20260811 22:18:48.568399 5946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:48.569489 5946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:48.570739 5946 hybrid_clock.cc:648] HybridClock initialized: now 1786486728570666 us; error 64 us; skew 500 ppm
I20260811 22:18:48.571146 5946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:48 dist-test-slave-jq4t krb5kdc[5517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486728, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:48.575564 5946 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:18:48.577014 5946 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:48.577157 5946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:48.577504 5946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:48.580736 5946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "887c344042424e028f786e452c62a39e"
format_stamp: "Formatted at 2026-08-11 22:18:48 on dist-test-slave-jq4t"
server_key: "75dc0332a5210cb13f56b2058b2500d1"
server_key_iv: "3203f262cd7935c50a4243784b90bc5c"
server_key_version: "encryptionkey@0"
I20260811 22:18:48.581615 5946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "887c344042424e028f786e452c62a39e"
format_stamp: "Formatted at 2026-08-11 22:18:48 on dist-test-slave-jq4t"
server_key: "75dc0332a5210cb13f56b2058b2500d1"
server_key_iv: "3203f262cd7935c50a4243784b90bc5c"
server_key_version: "encryptionkey@0"
I20260811 22:18:48.587073 5946 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:18:48.590927 5974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:48.592244 5946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:18:48.592453 5946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "887c344042424e028f786e452c62a39e"
format_stamp: "Formatted at 2026-08-11 22:18:48 on dist-test-slave-jq4t"
server_key: "75dc0332a5210cb13f56b2058b2500d1"
server_key_iv: "3203f262cd7935c50a4243784b90bc5c"
server_key_version: "encryptionkey@0"
I20260811 22:18:48.592787 5946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:48.605487 5946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:48.610276 5946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:48.610592 5946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:48.611701 5946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:48.613250 5946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:48.613379 5946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:48.613517 5946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:48.613610 5946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:48.655040 5946 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:33521
I20260811 22:18:48.655144 6222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:33521 every 8 connection(s)
I20260811 22:18:48.656015 5946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:18:48.660728 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 5946
I20260811 22:18:48.660961 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:18:48.661572 30612 external_mini_cluster.cc:1428] Setting key 5ff629188f0b269b157c982fa10f2afb
Aug 11 22:18:48 dist-test-slave-jq4t krb5kdc[5517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486728, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:48.682767 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:48.658666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:56793 (local address 127.29.229.62:43515)
0811 22:18:48.659251 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:48.659255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:48.660449 (+ 1194us) server_negotiation.cc:407] Connection header received
0811 22:18:48.663025 (+ 2576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:48.663030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:48.663107 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:48.663240 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:48.665508 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:48.666243 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:48.668897 (+ 2654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:48.669347 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:48.669462 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:48.674360 (+ 4898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:48.674365 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:48.674369 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:48.674406 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:48.678801 (+ 4395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:48.679749 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:48.679758 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:48.679765 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:48.679980 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:48.680471 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:48.680510 (+ 39us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:48.680516 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:48.680956 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:48.681125 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:48.681980 (+ 855us) server_negotiation.cc:299] Negotiation successful
0811 22:18:48.682266 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":345,"threads_started":1}
I20260811 22:18:48.683507 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:48.659352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43515 (local address 127.29.229.2:56793)
0811 22:18:48.660249 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:48.660302 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:48.662707 (+ 2405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:48.663506 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:48.663548 (+ 42us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:48.664365 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:48.665265 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:48.665305 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:48.666433 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:48.666442 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:48.668689 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:48.668708 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:48.669603 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:48.669612 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:48.669957 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:48.670833 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:48.670869 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:48.674062 (+ 3193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:48.679034 (+ 4972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:48.679044 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:48.679059 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:48.679576 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:48.680118 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:48.680122 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:48.680124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:48.680343 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:48.681130 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:48.681137 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:48.681523 (+ 386us) client_negotiation.cc:769] Sending connection context
0811 22:18:48.681776 (+ 253us) client_negotiation.cc:240] Negotiation successful
0811 22:18:48.682211 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":322,"threads_started":1}
I20260811 22:18:48.685201 6225 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43515
I20260811 22:18:48.685551 6225 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:48.686223 6225 heartbeater.cc:511] Master 127.29.229.62:43515 requested a full tablet report, sending...
I20260811 22:18:48.688066 5606 ts_manager.cc:194] Registered new tserver with Master: 887c344042424e028f786e452c62a39e (127.29.229.2:33521)
I20260811 22:18:48.688962 5606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:56793
WARNING: no policy specified for kudu/127.29.229.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
I20260811 22:18:48.725651 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43515
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:44739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:49.086290 6237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:49.086679 6237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:49.086876 6237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:49.095039 6237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:49.095207 6237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:49.095396 6237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:18:49.105877 6237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:44739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:49.107312 6237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:49.108755 6237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:49.122221 6248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:49.122390 6247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:49.122325 6252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:49.123428 6237 server_base.cc:1034] running on GCE node
I20260811 22:18:49.124082 6237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:49.125176 6237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:49.126405 6237 hybrid_clock.cc:648] HybridClock initialized: now 1786486729126323 us; error 68 us; skew 500 ppm
I20260811 22:18:49.126775 6237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:49 dist-test-slave-jq4t krb5kdc[5517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486729, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:49.130244 5939 heartbeater.cc:503] Master 127.29.229.62:43515 was elected leader, sending a full tablet report...
I20260811 22:18:49.130555 6237 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:18:49.132076 6237 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:49.132216 6237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:49.132658 6237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:49.136023 6237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "2064038469004d5fb3cda30b6dd81418"
format_stamp: "Formatted at 2026-08-11 22:18:49 on dist-test-slave-jq4t"
server_key: "49836059e0836556295865d06bb947fd"
server_key_iv: "5d1665f7330c69fb407a235b9cddfcb1"
server_key_version: "encryptionkey@0"
I20260811 22:18:49.137128 6237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "2064038469004d5fb3cda30b6dd81418"
format_stamp: "Formatted at 2026-08-11 22:18:49 on dist-test-slave-jq4t"
server_key: "49836059e0836556295865d06bb947fd"
server_key_iv: "5d1665f7330c69fb407a235b9cddfcb1"
server_key_version: "encryptionkey@0"
I20260811 22:18:49.143025 6237 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 22:18:49.146955 6266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:49.148418 6237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:18:49.148645 6237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "2064038469004d5fb3cda30b6dd81418"
format_stamp: "Formatted at 2026-08-11 22:18:49 on dist-test-slave-jq4t"
server_key: "49836059e0836556295865d06bb947fd"
server_key_iv: "5d1665f7330c69fb407a235b9cddfcb1"
server_key_version: "encryptionkey@0"
I20260811 22:18:49.148864 6237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:49.167372 6237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:49.172722 6237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:49.173045 6237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:49.174268 6237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:49.176026 6237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:49.176128 6237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:49.176235 6237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:49.176342 6237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:49.218371 6237 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:42959
I20260811 22:18:49.218431 6476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:42959 every 8 connection(s)
I20260811 22:18:49.219295 6237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:18:49.226702 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 6237
I20260811 22:18:49.226933 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:18:49.227389 30612 external_mini_cluster.cc:1428] Setting key 63a94a73caa94f7c03724ffa41936dd7
Aug 11 22:18:49 dist-test-slave-jq4t krb5kdc[5517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486729, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:49.240288 6479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.222157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:57547 (local address 127.29.229.62:43515)
0811 22:18:49.222648 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:49.222652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:49.223578 (+ 926us) server_negotiation.cc:407] Connection header received
0811 22:18:49.225568 (+ 1990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:49.225574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:49.225681 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:49.225812 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:49.227500 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.228306 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.230550 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.231049 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.231155 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.234873 (+ 3718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:49.234878 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:49.234881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:49.234913 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:49.237548 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:49.238269 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:49.238273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:49.238275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:49.238392 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:49.238774 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:49.238776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:49.238778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:49.239031 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:49.239152 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:49.239835 (+ 683us) server_negotiation.cc:299] Negotiation successful
0811 22:18:49.240014 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":243,"threads_started":1}
I20260811 22:18:49.241506 6481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.222614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43515 (local address 127.29.229.3:57547)
0811 22:18:49.223394 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:49.223445 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:49.225277 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:49.226055 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:49.226063 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:49.226616 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:49.227312 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:49.227330 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.228549 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.228554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:49.230378 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:49.230395 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.231266 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.231271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:49.231461 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.232212 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:18:49.232235 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:49.234665 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:49.237688 (+ 3023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:49.237695 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:49.237707 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:49.238152 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:49.238495 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:18:49.238499 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:18:49.238501 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:18:49.238676 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:18:49.239160 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:49.239167 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:49.239484 (+ 317us) client_negotiation.cc:769] Sending connection context
0811 22:18:49.239718 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 22:18:49.240139 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":347,"threads_started":1}
I20260811 22:18:49.243153 6477 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43515
I20260811 22:18:49.243512 6477 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:49.244187 6477 heartbeater.cc:511] Master 127.29.229.62:43515 requested a full tablet report, sending...
I20260811 22:18:49.245873 5606 ts_manager.cc:194] Registered new tserver with Master: 2064038469004d5fb3cda30b6dd81418 (127.29.229.3:42959)
I20260811 22:18:49.246829 5606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:57547
I20260811 22:18:49.256428 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:18:49.276314 6479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.262397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56900 (local address 127.29.229.62:43515)
0811 22:18:49.262694 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:49.262703 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:49.263077 (+ 374us) server_negotiation.cc:407] Connection header received
0811 22:18:49.263300 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:49.263310 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:49.263379 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:49.263493 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:49.265085 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.265842 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.267161 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.267585 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.267734 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.270421 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:49.270426 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:49.270429 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:49.270467 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:49.273505 (+ 3038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:49.274225 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:49.274238 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:49.274241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:49.274368 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:49.274758 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:49.274766 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:49.274770 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:49.275123 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:49.275407 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:49.275735 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 22:18:49.275984 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260811 22:18:49.283110 5606 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:18:49.291582 5606 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:18:49.329774 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.306028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58204 (local address 127.29.229.3:42959)
0811 22:18:49.321346 (+ 15318us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:49.321358 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:49.321421 (+ 63us) server_negotiation.cc:407] Connection header received
0811 22:18:49.321531 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:49.321538 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:49.321819 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:49.322743 (+ 924us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:49.323076 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.324352 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.327074 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.328413 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.328657 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.328923 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:49.329208 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 22:18:49.329379 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15074,"thread_start_us":14890,"threads_started":1}
I20260811 22:18:49.329774 6503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.305689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:42959 (local address 127.0.0.1:58204)
0811 22:18:49.319674 (+ 13985us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:49.319697 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:49.319836 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:49.322106 (+ 2270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:49.322114 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:49.322146 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:49.322568 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:49.322582 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.324538 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.324543 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:49.326872 (+ 2329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:49.326888 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.328716 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.328723 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:49.328927 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.328987 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 22:18:49.329151 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 22:18:49.329369 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13843,"thread_start_us":13673,"threads_started":1}
I20260811 22:18:49.331259 6497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.305281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:43947 (local address 127.0.0.1:53382)
0811 22:18:49.318694 (+ 13413us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:49.318747 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:49.318985 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:49.322405 (+ 3420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:49.322412 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:49.322977 (+ 565us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:49.323439 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:49.323452 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.325452 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.325460 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:49.327755 (+ 2295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:49.327771 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.330397 (+ 2626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.330404 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:49.330579 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.330609 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:18:49.330705 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 22:18:49.330868 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13033,"mutex_wait_us":3,"thread_start_us":12798,"threads_started":1}
I20260811 22:18:49.331293 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.305451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53382 (local address 127.29.229.1:43947)
0811 22:18:49.321861 (+ 16410us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:49.321869 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:49.321910 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:18:49.322006 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:49.322014 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:49.322188 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:49.322397 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:49.323663 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.325316 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.327947 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.329768 (+ 1821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.330360 (+ 592us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.330588 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:49.330765 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 22:18:49.330957 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16194,"thread_start_us":15959,"threads_started":1}
I20260811 22:18:49.332163 6509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.305299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:33521 (local address 127.0.0.1:50306)
0811 22:18:49.320459 (+ 15160us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:49.320510 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:49.320631 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:49.322729 (+ 2098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:49.322733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:49.322757 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:49.323322 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:49.323340 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.325392 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.325397 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:49.327548 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:49.327559 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.331426 (+ 3867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.331433 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:49.331637 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.331670 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 22:18:49.331764 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 22:18:49.331904 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15007,"mutex_wait_us":1144,"thread_start_us":13715,"threads_started":1}
I20260811 22:18:49.333091 6506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.305761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50306 (local address 127.29.229.2:33521)
0811 22:18:49.322141 (+ 16380us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:49.322149 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:49.322177 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:18:49.322278 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:49.322285 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:49.322510 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:49.322712 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:49.323564 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.325227 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.327709 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.330924 (+ 3215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.331083 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.331256 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:49.332405 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0811 22:18:49.332705 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16212,"thread_start_us":16032,"threads_started":1}
I20260811 22:18:49.334170 6360 tablet_service.cc:1467] Processing CreateTablet for tablet c8358de169964267b445677561cd2773 (DEFAULT_TABLE table=test-table [id=365aa8cdb08840298a81df3f0369edb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:49.335053 5827 tablet_service.cc:1467] Processing CreateTablet for tablet c8358de169964267b445677561cd2773 (DEFAULT_TABLE table=test-table [id=365aa8cdb08840298a81df3f0369edb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:49.335866 6360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8358de169964267b445677561cd2773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:49.336249 5827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8358de169964267b445677561cd2773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:49.336941 6080 tablet_service.cc:1467] Processing CreateTablet for tablet c8358de169964267b445677561cd2773 (DEFAULT_TABLE table=test-table [id=365aa8cdb08840298a81df3f0369edb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:49.338699 6080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8358de169964267b445677561cd2773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:49.348562 6511 tablet_bootstrap.cc:492] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe: Bootstrap starting.
I20260811 22:18:49.352200 6511 tablet_bootstrap.cc:654] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:49.353471 6511 log.cc:824] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:49.354965 6512 tablet_bootstrap.cc:492] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418: Bootstrap starting.
I20260811 22:18:49.356837 6514 tablet_bootstrap.cc:492] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e: Bootstrap starting.
I20260811 22:18:49.357054 6511 tablet_bootstrap.cc:492] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe: No bootstrap required, opened a new log
I20260811 22:18:49.357386 6511 ts_tablet_manager.cc:1397] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 22:18:49.360260 6514 tablet_bootstrap.cc:654] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:49.360335 6512 tablet_bootstrap.cc:654] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:49.361557 6514 log.cc:824] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:49.361689 6512 log.cc:824] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:49.369277 6511 raft_consensus.cc:348] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb73c6a6cc542648d67680df8471efe" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43947 } } peers { permanent_uuid: "887c344042424e028f786e452c62a39e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33521 } } peers { permanent_uuid: "2064038469004d5fb3cda30b6dd81418" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42959 } }
I20260811 22:18:49.369791 6511 raft_consensus.cc:374] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:49.369900 6511 raft_consensus.cc:729] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eb73c6a6cc542648d67680df8471efe, State: Initialized, Role: FOLLOWER
I20260811 22:18:49.370761 6511 consensus_queue.cc:260] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb73c6a6cc542648d67680df8471efe" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43947 } } peers { permanent_uuid: "887c344042424e028f786e452c62a39e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33521 } } peers { permanent_uuid: "2064038469004d5fb3cda30b6dd81418" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42959 } }
I20260811 22:18:49.371356 6514 tablet_bootstrap.cc:492] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e: No bootstrap required, opened a new log
I20260811 22:18:49.371500 6512 tablet_bootstrap.cc:492] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418: No bootstrap required, opened a new log
I20260811 22:18:49.371810 6514 ts_tablet_manager.cc:1397] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.000s
I20260811 22:18:49.371814 6512 ts_tablet_manager.cc:1397] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.007s
I20260811 22:18:49.372823 6511 ts_tablet_manager.cc:1428] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe: Time spent starting tablet: real 0.015s user 0.011s sys 0.000s
I20260811 22:18:49.381742 6514 raft_consensus.cc:348] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb73c6a6cc542648d67680df8471efe" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43947 } } peers { permanent_uuid: "887c344042424e028f786e452c62a39e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33521 } } peers { permanent_uuid: "2064038469004d5fb3cda30b6dd81418" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42959 } }
I20260811 22:18:49.382313 6514 raft_consensus.cc:374] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:49.382426 6514 raft_consensus.cc:729] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 887c344042424e028f786e452c62a39e, State: Initialized, Role: FOLLOWER
I20260811 22:18:49.382201 6512 raft_consensus.cc:348] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb73c6a6cc542648d67680df8471efe" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43947 } } peers { permanent_uuid: "887c344042424e028f786e452c62a39e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33521 } } peers { permanent_uuid: "2064038469004d5fb3cda30b6dd81418" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42959 } }
I20260811 22:18:49.382722 6512 raft_consensus.cc:374] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:49.382846 6512 raft_consensus.cc:729] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2064038469004d5fb3cda30b6dd81418, State: Initialized, Role: FOLLOWER
I20260811 22:18:49.383296 6514 consensus_queue.cc:260] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb73c6a6cc542648d67680df8471efe" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43947 } } peers { permanent_uuid: "887c344042424e028f786e452c62a39e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33521 } } peers { permanent_uuid: "2064038469004d5fb3cda30b6dd81418" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42959 } }
I20260811 22:18:49.383628 6512 consensus_queue.cc:260] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb73c6a6cc542648d67680df8471efe" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43947 } } peers { permanent_uuid: "887c344042424e028f786e452c62a39e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33521 } } peers { permanent_uuid: "2064038469004d5fb3cda30b6dd81418" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42959 } }
I20260811 22:18:49.385851 6514 ts_tablet_manager.cc:1428] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260811 22:18:49.386689 6225 heartbeater.cc:503] Master 127.29.229.62:43515 was elected leader, sending a full tablet report...
I20260811 22:18:49.389060 6477 heartbeater.cc:503] Master 127.29.229.62:43515 was elected leader, sending a full tablet report...
I20260811 22:18:49.390772 6512 ts_tablet_manager.cc:1428] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418: Time spent starting tablet: real 0.019s user 0.017s sys 0.000s
I20260811 22:18:49.398928 6521 raft_consensus.cc:482] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:18:49.399392 6521 raft_consensus.cc:504] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb73c6a6cc542648d67680df8471efe" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43947 } } peers { permanent_uuid: "887c344042424e028f786e452c62a39e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33521 } } peers { permanent_uuid: "2064038469004d5fb3cda30b6dd81418" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42959 } }
I20260811 22:18:49.401825 6521 leader_election.cc:288] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2eb73c6a6cc542648d67680df8471efe (127.29.229.1:43947), 2064038469004d5fb3cda30b6dd81418 (127.29.229.3:42959)
W20260811 22:18:49.408025 6226 tablet.cc:2369] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:18:49.410583 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.402634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:60073 (local address 127.29.229.3:42959)
0811 22:18:49.402846 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:49.402855 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:49.403405 (+ 550us) server_negotiation.cc:407] Connection header received
0811 22:18:49.403722 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:49.403727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:49.403814 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:49.403865 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:49.404799 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.405941 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.408455 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.409504 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.409633 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.409688 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:49.410179 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 22:18:49.410314 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:18:49.410955 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.402119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:42959 (local address 127.29.229.2:60073)
0811 22:18:49.403276 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:49.403301 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:49.403540 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:49.404082 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:49.404087 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:49.404113 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:49.404627 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:49.404646 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.406093 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.406099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:49.408258 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:49.408273 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.409869 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.409873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:49.410009 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.410032 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:18:49.410149 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 22:18:49.410294 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":960,"mutex_wait_us":3,"thread_start_us":1127,"threads_started":1}
I20260811 22:18:49.412175 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.402408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:54861 (local address 127.29.229.1:43947)
0811 22:18:49.402665 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:49.402672 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:49.403640 (+ 968us) server_negotiation.cc:407] Connection header received
0811 22:18:49.403780 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:49.403785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:49.403870 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:49.403967 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:18:49.404896 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.405622 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.408871 (+ 3249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.410079 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.410205 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.410275 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:49.411722 (+ 1447us) server_negotiation.cc:299] Negotiation successful
0811 22:18:49.411912 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:18:49.412210 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.402188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:43947 (local address 127.29.229.2:54861)
0811 22:18:49.403528 (+ 1340us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:49.403544 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:49.403658 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:49.404023 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:49.404031 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:49.404063 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:18:49.404674 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:49.404686 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.405761 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.405768 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:49.408676 (+ 2908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:49.408691 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.410496 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.410502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:49.410674 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.411570 (+ 896us) client_negotiation.cc:769] Sending connection context
0811 22:18:49.411661 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:18:49.411802 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1210,"mutex_wait_us":28,"thread_start_us":668,"threads_started":1}
I20260811 22:18:49.412532 6389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8358de169964267b445677561cd2773" candidate_uuid: "887c344042424e028f786e452c62a39e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2064038469004d5fb3cda30b6dd81418" is_pre_election: true
I20260811 22:18:49.413159 6389 raft_consensus.cc:2454] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 887c344042424e028f786e452c62a39e in term 0.
I20260811 22:18:49.413190 5867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8358de169964267b445677561cd2773" candidate_uuid: "887c344042424e028f786e452c62a39e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eb73c6a6cc542648d67680df8471efe" is_pre_election: true
I20260811 22:18:49.413761 5867 raft_consensus.cc:2454] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 887c344042424e028f786e452c62a39e in term 0.
I20260811 22:18:49.413883 5975 leader_election.cc:302] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2064038469004d5fb3cda30b6dd81418, 887c344042424e028f786e452c62a39e; no voters:
I20260811 22:18:49.414353 6521 raft_consensus.cc:2781] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:18:49.414546 6521 raft_consensus.cc:482] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:18:49.414674 6521 raft_consensus.cc:3037] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:49.417182 6521 raft_consensus.cc:504] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb73c6a6cc542648d67680df8471efe" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43947 } } peers { permanent_uuid: "887c344042424e028f786e452c62a39e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33521 } } peers { permanent_uuid: "2064038469004d5fb3cda30b6dd81418" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42959 } }
I20260811 22:18:49.417882 6521 leader_election.cc:288] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [CANDIDATE]: Term 1 election: Requested vote from peers 2eb73c6a6cc542648d67680df8471efe (127.29.229.1:43947), 2064038469004d5fb3cda30b6dd81418 (127.29.229.3:42959)
I20260811 22:18:49.418339 5867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8358de169964267b445677561cd2773" candidate_uuid: "887c344042424e028f786e452c62a39e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eb73c6a6cc542648d67680df8471efe"
I20260811 22:18:49.418560 5867 raft_consensus.cc:3037] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:49.418545 6389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8358de169964267b445677561cd2773" candidate_uuid: "887c344042424e028f786e452c62a39e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2064038469004d5fb3cda30b6dd81418"
I20260811 22:18:49.418777 6389 raft_consensus.cc:3037] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:49.421032 5867 raft_consensus.cc:2454] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 887c344042424e028f786e452c62a39e in term 1.
I20260811 22:18:49.421279 6389 raft_consensus.cc:2454] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 887c344042424e028f786e452c62a39e in term 1.
I20260811 22:18:49.421530 5980 leader_election.cc:302] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2eb73c6a6cc542648d67680df8471efe, 887c344042424e028f786e452c62a39e; no voters:
I20260811 22:18:49.421901 6521 raft_consensus.cc:2781] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:49.422432 6521 raft_consensus.cc:686] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [term 1 LEADER]: Becoming Leader. State: Replica: 887c344042424e028f786e452c62a39e, State: Running, Role: LEADER
I20260811 22:18:49.423122 6521 consensus_queue.cc:237] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb73c6a6cc542648d67680df8471efe" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43947 } } peers { permanent_uuid: "887c344042424e028f786e452c62a39e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33521 } } peers { permanent_uuid: "2064038469004d5fb3cda30b6dd81418" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42959 } }
I20260811 22:18:49.428086 5606 catalog_manager.cc:5495] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e reported cstate change: term changed from 0 to 1, leader changed from <none> to 887c344042424e028f786e452c62a39e (127.29.229.2). New cstate: current_term: 1 leader_uuid: "887c344042424e028f786e452c62a39e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb73c6a6cc542648d67680df8471efe" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "887c344042424e028f786e452c62a39e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2064038469004d5fb3cda30b6dd81418" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42959 } health_report { overall_health: UNKNOWN } } }
I20260811 22:18:49.444809 6506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.439444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50312 (local address 127.29.229.2:33521)
0811 22:18:49.439636 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:49.439641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:49.439741 (+ 100us) server_negotiation.cc:407] Connection header received
0811 22:18:49.439975 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:49.439980 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:49.440068 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:49.440227 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:18:49.440658 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.441370 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.442739 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.443144 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.443237 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.443713 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:18:49.444224 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:18:49.444333 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:49.444463 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 22:18:49.444592 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:18:49.453922 5867 raft_consensus.cc:1261] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe [term 1 FOLLOWER]: Refusing update from remote peer 887c344042424e028f786e452c62a39e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:49.454591 6389 raft_consensus.cc:1261] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418 [term 1 FOLLOWER]: Refusing update from remote peer 887c344042424e028f786e452c62a39e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:49.454950 6526 consensus_queue.cc:1035] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2eb73c6a6cc542648d67680df8471efe" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:49.455499 6521 consensus_queue.cc:1035] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2064038469004d5fb3cda30b6dd81418" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:18:49.471051 6478 tablet.cc:2369] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:18:49.486377 5606 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:18:49.486699 5606 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:18:49.490733 5606 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cf72ac26cc5041d1af24b7e939c2450c: Sending DeleteTablet for 3 replicas of tablet c8358de169964267b445677561cd2773
I20260811 22:18:49.492015 5827 tablet_service.cc:1514] Processing DeleteTablet for tablet c8358de169964267b445677561cd2773 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:49 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:53382
I20260811 22:18:49.492066 6080 tablet_service.cc:1514] Processing DeleteTablet for tablet c8358de169964267b445677561cd2773 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:49 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:50306
I20260811 22:18:49.492178 6360 tablet_service.cc:1514] Processing DeleteTablet for tablet c8358de169964267b445677561cd2773 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:49 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:58204
I20260811 22:18:49.494410 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 5713
I20260811 22:18:49.501084 6551 tablet_replica.cc:330] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe: stopping tablet replica
I20260811 22:18:49.501552 6551 raft_consensus.cc:2229] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:18:49.502138 6551 raft_consensus.cc:2258] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:49.502992 6553 tablet_replica.cc:330] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e: stopping tablet replica
I20260811 22:18:49.503490 6553 raft_consensus.cc:2229] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:18:49.504225 6553 raft_consensus.cc:2258] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:49.505338 6551 ts_tablet_manager.cc:1905] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:49.507707 6553 ts_tablet_manager.cc:1905] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:49.515998 6551 ts_tablet_manager.cc:1918] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:18:49.516309 6551 log.cc:1196] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-0/wal/wals/c8358de169964267b445677561cd2773
I20260811 22:18:49.517078 6551 ts_tablet_manager.cc:1939] T c8358de169964267b445677561cd2773 P 2eb73c6a6cc542648d67680df8471efe: Deleting consensus metadata
I20260811 22:18:49.519701 5577 catalog_manager.cc:4841] TS 2eb73c6a6cc542648d67680df8471efe (127.29.229.1:43947): tablet c8358de169964267b445677561cd2773 (table test-table [id=365aa8cdb08840298a81df3f0369edb8]) successfully deleted
I20260811 22:18:49.520713 6555 tablet_replica.cc:330] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418: stopping tablet replica
I20260811 22:18:49.521234 6555 raft_consensus.cc:2229] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:18:49.521795 6555 raft_consensus.cc:2258] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:49.522637 6553 ts_tablet_manager.cc:1918] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:18:49.522876 6553 log.cc:1196] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/c8358de169964267b445677561cd2773
I20260811 22:18:49.523581 6553 ts_tablet_manager.cc:1939] T c8358de169964267b445677561cd2773 P 887c344042424e028f786e452c62a39e: Deleting consensus metadata
I20260811 22:18:49.524507 6555 ts_tablet_manager.cc:1905] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 22:18:49 dist-test-slave-jq4t krb5kdc[5517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:18:49.525619 5576 catalog_manager.cc:4841] TS 887c344042424e028f786e452c62a39e (127.29.229.2:33521): tablet c8358de169964267b445677561cd2773 (table test-table [id=365aa8cdb08840298a81df3f0369edb8]) successfully deleted
I20260811 22:18:49.530429 6555 ts_tablet_manager.cc:1918] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:18:49.530656 6555 log.cc:1196] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/c8358de169964267b445677561cd2773
I20260811 22:18:49.531175 6555 ts_tablet_manager.cc:1939] T c8358de169964267b445677561cd2773 P 2064038469004d5fb3cda30b6dd81418: Deleting consensus metadata
I20260811 22:18:49.533414 5574 catalog_manager.cc:4841] TS 2064038469004d5fb3cda30b6dd81418 (127.29.229.3:42959): tablet c8358de169964267b445677561cd2773 (table test-table [id=365aa8cdb08840298a81df3f0369edb8]) successfully deleted
I20260811 22:18:49.534909 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.495937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53392 (local address 127.29.229.1:43947)
0811 22:18:49.496142 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:49.496148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:49.500800 (+ 4652us) server_negotiation.cc:407] Connection header received
0811 22:18:49.501861 (+ 1061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:49.501866 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:49.501949 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:49.502060 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:49.512637 (+ 10577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.513929 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.516557 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.517263 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.517375 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.526548 (+ 9173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:49.526571 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:49.526591 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:49.526645 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:49.531129 (+ 4484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:49.532222 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:49.532233 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:49.532237 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:49.532448 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:49.532944 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:49.532949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:49.532952 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:49.533547 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:49.533996 (+ 449us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:49.534230 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 22:18:49.534496 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:18:49.535698 5930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:49.648130 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 5713
I20260811 22:18:49.679235 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 5946
Aug 11 22:18:49 dist-test-slave-jq4t krb5kdc[5517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.2@KRBTEST.COM
I20260811 22:18:49.697276 6506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.680029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50320 (local address 127.29.229.2:33521)
0811 22:18:49.680223 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:49.680229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:49.680354 (+ 125us) server_negotiation.cc:407] Connection header received
0811 22:18:49.680623 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:49.680627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:49.680716 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:49.680839 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:49.682073 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.682921 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.684167 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.684833 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.684980 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.689441 (+ 4461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:49.689454 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:49.689473 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:49.689517 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:49.693073 (+ 3556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:49.694088 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:49.694100 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:49.694102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:49.694388 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:49.694961 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:49.694966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:49.694968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:49.695807 (+ 839us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:49.696121 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:49.696561 (+ 440us) server_negotiation.cc:299] Negotiation successful
0811 22:18:49.696814 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:18:49.698146 6211 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:49.819674 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 5946
I20260811 22:18:49.850649 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 6237
Aug 11 22:18:49 dist-test-slave-jq4t krb5kdc[5517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.3@KRBTEST.COM
I20260811 22:18:49.869560 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:49.851630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58210 (local address 127.29.229.3:42959)
0811 22:18:49.852020 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:49.852025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:49.852054 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:18:49.852189 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:49.852194 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:49.852312 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:49.852425 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:49.854070 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.854973 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.856244 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:49.856843 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:49.856968 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:49.862138 (+ 5170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:49.862156 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:49.862174 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:49.862216 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:49.865880 (+ 3664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:49.867125 (+ 1245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:49.867138 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:49.867141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:49.867328 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:49.867832 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:49.867850 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:49.867860 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:49.868281 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:49.868613 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:49.869049 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 22:18:49.869275 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262}
I20260811 22:18:49.870635 6468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:49.964351 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 6237
I20260811 22:18:49.995864 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 5543
I20260811 22:18:49.997437 5656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:50.120802 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 5543
2026-08-11T22:18:50Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4883 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T22:18:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:18:50Z Disabled control of system clock
I20260811 22:18:50.215071 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:41453
--webserver_interface=127.29.229.62
--webserver_port=0
--builtin_ntp_servers=127.29.229.20:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:41453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 22:18:50.579636 6578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:50.579990 6578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:50.580154 6578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:50.587512 6578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:50.587628 6578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:50.587718 6578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:50.587822 6578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:50.587888 6578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:18:50.597415 6578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:41453
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:41453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:50.598778 6578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:50.600697 6578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:50.613322 6588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:50.613330 6592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:50.614068 6589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:50.614475 6578 server_base.cc:1034] running on GCE node
I20260811 22:18:50.615393 6578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:50.616963 6578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:50.618237 6578 hybrid_clock.cc:648] HybridClock initialized: now 1786486730618187 us; error 62 us; skew 500 ppm
I20260811 22:18:50.618637 6578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:18:50.624614 6578 webserver.cc:460] Webserver started at http://127.29.229.62:36895/ using document root <none> and password file <none>
I20260811 22:18:50.625384 6578 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:50.625530 6578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:50.625875 6578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:50.629004 6578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "9a9a5b91174b41ec918839db1628c170"
format_stamp: "Formatted at 2026-08-11 22:18:50 on dist-test-slave-jq4t"
server_key: "1a1b237dd47f525fcc32750e1c8df26f"
server_key_iv: "527aa43c0d03a27368d53dc615c57cde"
server_key_version: "encryptionkey@0"
I20260811 22:18:50.629837 6578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "9a9a5b91174b41ec918839db1628c170"
format_stamp: "Formatted at 2026-08-11 22:18:50 on dist-test-slave-jq4t"
server_key: "1a1b237dd47f525fcc32750e1c8df26f"
server_key_iv: "527aa43c0d03a27368d53dc615c57cde"
server_key_version: "encryptionkey@0"
I20260811 22:18:50.635512 6578 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:18:50.639194 6606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:50.640744 6578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:18:50.640959 6578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "9a9a5b91174b41ec918839db1628c170"
format_stamp: "Formatted at 2026-08-11 22:18:50 on dist-test-slave-jq4t"
server_key: "1a1b237dd47f525fcc32750e1c8df26f"
server_key_iv: "527aa43c0d03a27368d53dc615c57cde"
server_key_version: "encryptionkey@0"
I20260811 22:18:50.641211 6578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:50.646956 6578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:50.647747 6578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:50.648072 6578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:50.667675 6578 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:41453
I20260811 22:18:50.667768 6710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:41453 every 8 connection(s)
I20260811 22:18:50.669180 6578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:50.671816 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 6578
I20260811 22:18:50.672040 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:18:50.673064 30612 external_mini_cluster.cc:1428] Setting key 30310957fe557875e6185f2436a7d845
I20260811 22:18:50.674031 6712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:50.682557 6715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:50.675402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38370 (local address 127.29.229.62:41453)
0811 22:18:50.676584 (+ 1182us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:50.676595 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:50.676640 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:18:50.678890 (+ 2250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:50.678913 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:50.679324 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:50.679747 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:50.680902 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:50.680909 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:50.680927 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:18:50.680942 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:50.681004 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:18:50.681021 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:50.681289 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:50.681459 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 22:18:50.681708 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":424,"threads_started":1}
I20260811 22:18:50.687305 6712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170: Bootstrap starting.
I20260811 22:18:50.690599 6712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:50.691812 6712 log.cc:824] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:50.695081 6712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170: No bootstrap required, opened a new log
I20260811 22:18:50.702870 6712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9a5b91174b41ec918839db1628c170" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41453 } }
I20260811 22:18:50.703395 6712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:50.703480 6712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a9a5b91174b41ec918839db1628c170, State: Initialized, Role: FOLLOWER
I20260811 22:18:50.704224 6712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9a5b91174b41ec918839db1628c170" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41453 } }
I20260811 22:18:50.704525 6712 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:18:50.704663 6712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:18:50.704807 6712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:50.707129 6712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9a5b91174b41ec918839db1628c170" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41453 } }
I20260811 22:18:50.707676 6712 leader_election.cc:302] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a9a5b91174b41ec918839db1628c170; no voters:
I20260811 22:18:50.708468 6712 leader_election.cc:288] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:18:50.708644 6720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:50.709481 6720 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170 [term 1 LEADER]: Becoming Leader. State: Replica: 9a9a5b91174b41ec918839db1628c170, State: Running, Role: LEADER
I20260811 22:18:50.710013 6720 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9a5b91174b41ec918839db1628c170" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41453 } }
I20260811 22:18:50.710779 6712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:50.713039 6722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a9a5b91174b41ec918839db1628c170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9a5b91174b41ec918839db1628c170" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41453 } } }
I20260811 22:18:50.713362 6722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:50.714268 6733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:50.714077 6724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a9a5b91174b41ec918839db1628c170. Latest consensus state: current_term: 1 leader_uuid: "9a9a5b91174b41ec918839db1628c170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9a5b91174b41ec918839db1628c170" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41453 } } }
I20260811 22:18:50.715299 6724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:50.719208 6733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:50.728509 6733 catalog_manager.cc:1347] Generated new cluster ID: 97a7d598ff8b4308a04cdf22ed6893d3
I20260811 22:18:50.728628 6733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:50.749946 6733 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:18:50.750416 6733 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:50.758309 6733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170: Generated new TSK 0
I20260811 22:18:50.759366 6733 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:18:50.769387 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41453
--builtin_ntp_servers=127.29.229.20:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 22:18:51.133185 6748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:51.133496 6748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:51.133591 6748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:51.141016 6748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:51.141139 6748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:51.141309 6748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:18:51.150957 6748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41453
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:51.152740 6748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:51.154726 6748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:51.169683 6760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:51.169636 6757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:51.169723 6756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:51.170650 6748 server_base.cc:1034] running on GCE node
I20260811 22:18:51.171551 6748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:51.172709 6748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:51.173980 6748 hybrid_clock.cc:648] HybridClock initialized: now 1786486731173936 us; error 52 us; skew 500 ppm
I20260811 22:18:51.174386 6748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:18:51.177474 6748 webserver.cc:460] Webserver started at http://127.29.229.1:39145/ using document root <none> and password file <none>
I20260811 22:18:51.178246 6748 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:51.178391 6748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:51.178774 6748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:51.182138 6748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "e8f02d604d4c4d73ae2b9d64e9d3693e"
format_stamp: "Formatted at 2026-08-11 22:18:51 on dist-test-slave-jq4t"
server_key: "d629f8cb49d83ee6ebe93f1fb9eeb0a5"
server_key_iv: "17f3a69b65c6da8a0aca306e8fde0f10"
server_key_version: "encryptionkey@0"
I20260811 22:18:51.183028 6748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "e8f02d604d4c4d73ae2b9d64e9d3693e"
format_stamp: "Formatted at 2026-08-11 22:18:51 on dist-test-slave-jq4t"
server_key: "d629f8cb49d83ee6ebe93f1fb9eeb0a5"
server_key_iv: "17f3a69b65c6da8a0aca306e8fde0f10"
server_key_version: "encryptionkey@0"
I20260811 22:18:51.189314 6748 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 22:18:51.194847 6772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:51.197183 6748 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:18:51.197436 6748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "e8f02d604d4c4d73ae2b9d64e9d3693e"
format_stamp: "Formatted at 2026-08-11 22:18:51 on dist-test-slave-jq4t"
server_key: "d629f8cb49d83ee6ebe93f1fb9eeb0a5"
server_key_iv: "17f3a69b65c6da8a0aca306e8fde0f10"
server_key_version: "encryptionkey@0"
I20260811 22:18:51.197715 6748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:51.204351 6748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:51.205315 6748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:51.205633 6748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:51.206674 6748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:51.209267 6748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:51.209447 6748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:51.209618 6748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:51.209733 6748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:51.250661 6748 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:42629
I20260811 22:18:51.250723 6984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:42629 every 8 connection(s)
I20260811 22:18:51.252069 6748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:18:51.258414 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 6748
I20260811 22:18:51.258631 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:18:51.259320 30612 external_mini_cluster.cc:1428] Setting key fc03d2e163f214ccc1c3153593c49a8f
I20260811 22:18:51.262871 6989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:51.254968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:35039 (local address 127.29.229.62:41453)
0811 22:18:51.255508 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:51.255512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:51.256645 (+ 1133us) server_negotiation.cc:407] Connection header received
0811 22:18:51.259065 (+ 2420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:51.259073 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:51.259179 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:51.259338 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:51.261523 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:51.261531 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:51.261535 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:18:51.261637 (+ 102us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:51.261822 (+ 185us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:18:51.261863 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:51.262047 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:51.262326 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 22:18:51.262417 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":293,"threads_started":1}
I20260811 22:18:51.263173 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41453
--builtin_ntp_servers=127.29.229.20:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 22:18:51.264060 6991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:51.255449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41453 (local address 127.29.229.1:35039)
0811 22:18:51.256405 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:51.256463 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:51.258556 (+ 2093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:51.259676 (+ 1120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:51.259688 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:51.259822 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:51.260918 (+ 1096us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:18:51.260922 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:51.260992 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:18:51.261261 (+ 269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:51.262003 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:51.262019 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:51.262031 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 22:18:51.262443 (+ 412us) client_negotiation.cc:240] Negotiation successful
0811 22:18:51.262903 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":307,"threads_started":1}
I20260811 22:18:51.265955 6985 heartbeater.cc:347] Connected to a master server at 127.29.229.62:41453
I20260811 22:18:51.266297 6985 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:51.266917 6985 heartbeater.cc:511] Master 127.29.229.62:41453 requested a full tablet report, sending...
I20260811 22:18:51.269310 6640 ts_manager.cc:194] Registered new tserver with Master: e8f02d604d4c4d73ae2b9d64e9d3693e (127.29.229.1:42629)
W20260811 22:18:51.630980 6992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:51.631280 6992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:51.631441 6992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:51.639240 6992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:51.639379 6992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:51.639568 6992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:18:51.650621 6992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41453
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:51.652072 6992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:51.654214 6992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:51.670445 7002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:51.670482 7007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:51.670542 7000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:51.672607 6992 server_base.cc:1034] running on GCE node
I20260811 22:18:51.673310 6992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:51.674363 6992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:51.675647 6992 hybrid_clock.cc:648] HybridClock initialized: now 1786486731675594 us; error 41 us; skew 500 ppm
I20260811 22:18:51.676045 6992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:18:51.678855 6992 webserver.cc:460] Webserver started at http://127.29.229.2:33125/ using document root <none> and password file <none>
I20260811 22:18:51.679797 6992 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:51.679957 6992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:51.680328 6992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:51.683512 6992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "c75c8b6c8be9464c822fd757cda5e6ce"
format_stamp: "Formatted at 2026-08-11 22:18:51 on dist-test-slave-jq4t"
server_key: "7e00aba10e520ef894046848d1fc402a"
server_key_iv: "aae7e2216a30f429b9863b3e1eb5fd4e"
server_key_version: "encryptionkey@0"
I20260811 22:18:51.684368 6992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "c75c8b6c8be9464c822fd757cda5e6ce"
format_stamp: "Formatted at 2026-08-11 22:18:51 on dist-test-slave-jq4t"
server_key: "7e00aba10e520ef894046848d1fc402a"
server_key_iv: "aae7e2216a30f429b9863b3e1eb5fd4e"
server_key_version: "encryptionkey@0"
I20260811 22:18:51.690816 6992 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 22:18:51.695509 7018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:51.697614 6992 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 22:18:51.697964 6992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "c75c8b6c8be9464c822fd757cda5e6ce"
format_stamp: "Formatted at 2026-08-11 22:18:51 on dist-test-slave-jq4t"
server_key: "7e00aba10e520ef894046848d1fc402a"
server_key_iv: "aae7e2216a30f429b9863b3e1eb5fd4e"
server_key_version: "encryptionkey@0"
I20260811 22:18:51.698238 6992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:51.704921 6992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:51.705837 6992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:51.706142 6992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:51.707204 6992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:51.708911 6992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:51.709046 6992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:51.709184 6992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:51.709285 6992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:51.750789 6992 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:36391
I20260811 22:18:51.750912 7217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:36391 every 8 connection(s)
I20260811 22:18:51.752187 6992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:18:51.755393 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 6992
I20260811 22:18:51.755643 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:18:51.756445 30612 external_mini_cluster.cc:1428] Setting key 542a818b247824d2be2e4262fbd66a00
I20260811 22:18:51.760197 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41453
--builtin_ntp_servers=127.29.229.20:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 22:18:51.764379 6989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:51.756183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:46041 (local address 127.29.229.62:41453)
0811 22:18:51.756399 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:51.756404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:51.758949 (+ 2545us) server_negotiation.cc:407] Connection header received
0811 22:18:51.760928 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:51.760937 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:51.761036 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:51.761158 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:51.763503 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:51.763510 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:51.763515 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:18:51.763542 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:51.763583 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:18:51.763596 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:51.763753 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:51.763987 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 22:18:51.764070 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:18:51.765799 7221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:51.757279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41453 (local address 127.29.229.2:46041)
0811 22:18:51.758732 (+ 1453us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:51.758797 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:51.760629 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:51.761666 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:51.761681 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:51.761838 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:51.762951 (+ 1113us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:18:51.762954 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:51.763011 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:18:51.763268 (+ 257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:51.763721 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:51.763735 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:51.763744 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 22:18:51.764102 (+ 358us) client_negotiation.cc:240] Negotiation successful
0811 22:18:51.764418 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"thread_start_us":386,"threads_started":1}
I20260811 22:18:51.767530 7218 heartbeater.cc:347] Connected to a master server at 127.29.229.62:41453
I20260811 22:18:51.767904 7218 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:51.768697 7218 heartbeater.cc:511] Master 127.29.229.62:41453 requested a full tablet report, sending...
I20260811 22:18:51.770483 6640 ts_manager.cc:194] Registered new tserver with Master: c75c8b6c8be9464c822fd757cda5e6ce (127.29.229.2:36391)
W20260811 22:18:52.113601 7222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:52.113901 7222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:52.114059 7222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:52.121459 7222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:52.121579 7222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:52.121776 7222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:18:52.131247 7222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41453
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:52.132618 7222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:52.134415 7222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:52.147984 7230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:52.148041 7231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:52.148520 7233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:52.148999 7222 server_base.cc:1034] running on GCE node
I20260811 22:18:52.150044 7222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:52.151562 7222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:52.152824 7222 hybrid_clock.cc:648] HybridClock initialized: now 1786486732152764 us; error 54 us; skew 500 ppm
I20260811 22:18:52.153218 7222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:18:52.155669 7222 webserver.cc:460] Webserver started at http://127.29.229.3:39439/ using document root <none> and password file <none>
I20260811 22:18:52.156387 7222 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:52.156531 7222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:52.156968 7222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:52.159940 7222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "979deba5352b4765bb4081a8c91df29c"
format_stamp: "Formatted at 2026-08-11 22:18:52 on dist-test-slave-jq4t"
server_key: "e56b4b208d86d4e930aa80394a521f9a"
server_key_iv: "d0cfaaa85e1666c8d5484dbf35752350"
server_key_version: "encryptionkey@0"
I20260811 22:18:52.160802 7222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "979deba5352b4765bb4081a8c91df29c"
format_stamp: "Formatted at 2026-08-11 22:18:52 on dist-test-slave-jq4t"
server_key: "e56b4b208d86d4e930aa80394a521f9a"
server_key_iv: "d0cfaaa85e1666c8d5484dbf35752350"
server_key_version: "encryptionkey@0"
I20260811 22:18:52.165970 7222 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 22:18:52.169400 7244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:52.171466 7222 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:18:52.171623 7222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "979deba5352b4765bb4081a8c91df29c"
format_stamp: "Formatted at 2026-08-11 22:18:52 on dist-test-slave-jq4t"
server_key: "e56b4b208d86d4e930aa80394a521f9a"
server_key_iv: "d0cfaaa85e1666c8d5484dbf35752350"
server_key_version: "encryptionkey@0"
I20260811 22:18:52.171847 7222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:52.178489 7222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:52.179421 7222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:52.179668 7222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:52.180943 7222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:18:52.182690 7222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:18:52.182793 7222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:52.182883 7222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:18:52.182948 7222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:52.222352 7222 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:45337
I20260811 22:18:52.222477 7448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:45337 every 8 connection(s)
I20260811 22:18:52.223719 7222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:18:52.230268 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 7222
I20260811 22:18:52.230468 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:18:52.230962 30612 external_mini_cluster.cc:1428] Setting key cf41610aa7acfec31a80aa13607835b0
I20260811 22:18:52.232698 6989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.226699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:39001 (local address 127.29.229.62:41453)
0811 22:18:52.226883 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:52.226887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:52.228343 (+ 1456us) server_negotiation.cc:407] Connection header received
0811 22:18:52.230176 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:52.230182 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:52.230272 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:52.230403 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:52.231866 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:52.231871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:52.231874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:18:52.231899 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:52.231955 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:18:52.231971 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:52.232095 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:52.232337 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 22:18:52.232417 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:18:52.233762 7451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.227240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41453 (local address 127.29.229.3:39001)
0811 22:18:52.228188 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:52.228224 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:52.229909 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:52.230655 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:52.230664 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:52.230751 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:52.231510 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:18:52.231512 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:52.231551 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:18:52.231669 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:52.232086 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:52.232095 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:52.232101 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:18:52.232431 (+ 330us) client_negotiation.cc:240] Negotiation successful
0811 22:18:52.232724 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":296,"threads_started":1}
I20260811 22:18:52.235054 7449 heartbeater.cc:347] Connected to a master server at 127.29.229.62:41453
I20260811 22:18:52.235350 7449 heartbeater.cc:464] Registering TS with master...
I20260811 22:18:52.235944 7449 heartbeater.cc:511] Master 127.29.229.62:41453 requested a full tablet report, sending...
I20260811 22:18:52.237386 6640 ts_manager.cc:194] Registered new tserver with Master: 979deba5352b4765bb4081a8c91df29c (127.29.229.3:45337)
I20260811 22:18:52.246526 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:18:52.254631 6989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.251531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38374 (local address 127.29.229.62:41453)
0811 22:18:52.251790 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:52.251800 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:52.252270 (+ 470us) server_negotiation.cc:407] Connection header received
0811 22:18:52.252439 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:52.252444 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:52.252564 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:52.252652 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:52.253923 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:52.253932 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:52.253937 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:18:52.253963 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:52.254009 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:18:52.254029 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:52.254171 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:52.254285 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 22:18:52.254350 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:18:52.258706 6640 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:18:52.265910 6640 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:18:52.272660 6985 heartbeater.cc:503] Master 127.29.229.62:41453 was elected leader, sending a full tablet report...
I20260811 22:18:52.290020 7466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.279624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:45337 (local address 127.0.0.1:48170)
0811 22:18:52.285202 (+ 5578us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:52.285227 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:52.285451 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:52.287472 (+ 2021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:52.287484 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:52.287599 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:52.288585 (+ 986us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:18:52.288588 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:52.288611 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:18:52.288678 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:52.289228 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:52.289239 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:52.289247 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:18:52.289434 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 22:18:52.289543 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5342,"thread_start_us":1821,"threads_started":1}
I20260811 22:18:52.291016 7468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.279082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:36391 (local address 127.0.0.1:47272)
0811 22:18:52.280369 (+ 1287us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:52.280415 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:52.280615 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:52.288978 (+ 8363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:52.288984 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:52.289008 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:52.289746 (+ 738us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:18:52.289750 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:52.289766 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:18:52.289819 (+ 53us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:52.290286 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:52.290294 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:52.290302 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:18:52.290444 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 22:18:52.290503 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":983,"thread_start_us":5618,"threads_started":1}
I20260811 22:18:52.291214 7470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.280774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48170 (local address 127.29.229.3:45337)
0811 22:18:52.281721 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:52.281731 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:52.285347 (+ 3616us) server_negotiation.cc:407] Connection header received
0811 22:18:52.285636 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:52.285645 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:52.285889 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:52.286052 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:52.288852 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:52.288862 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:52.288877 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:18:52.288898 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:52.288961 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:18:52.288986 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:52.289164 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:52.290683 (+ 1519us) server_negotiation.cc:299] Negotiation successful
0811 22:18:52.290789 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720,"thread_start_us":1303,"threads_started":1}
I20260811 22:18:52.291859 7467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.279289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47272 (local address 127.29.229.2:36391)
0811 22:18:52.288441 (+ 9152us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:52.288454 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:52.288515 (+ 61us) server_negotiation.cc:407] Connection header received
0811 22:18:52.288587 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:52.288593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:52.288800 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:52.288971 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:52.290038 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:52.290046 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:52.290057 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:18:52.290075 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:52.290134 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:18:52.290156 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:52.291399 (+ 1243us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:52.291468 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 22:18:52.291561 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8956,"thread_start_us":9136,"threads_started":1}
I20260811 22:18:52.293238 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.281227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53120 (local address 127.29.229.1:42629)
0811 22:18:52.288698 (+ 7471us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:52.288706 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:52.289228 (+ 522us) server_negotiation.cc:407] Connection header received
0811 22:18:52.289302 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:52.289310 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:52.289495 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:52.289639 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:52.291896 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:52.291904 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:52.291916 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:18:52.291934 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:52.291995 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:18:52.292015 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:52.292444 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:52.292617 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 22:18:52.292770 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6730,"thread_start_us":6570,"threads_started":1}
I20260811 22:18:52.295046 7325 tablet_service.cc:1467] Processing CreateTablet for tablet 59535e0cd069456da72ecdc23e7ab316 (DEFAULT_TABLE table=test-table [id=1c46333fd2e24e51ba161b6e14ba9c40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:52.295632 7096 tablet_service.cc:1467] Processing CreateTablet for tablet 59535e0cd069456da72ecdc23e7ab316 (DEFAULT_TABLE table=test-table [id=1c46333fd2e24e51ba161b6e14ba9c40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:52.296579 7325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59535e0cd069456da72ecdc23e7ab316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:52.296772 7469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.280046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:42629 (local address 127.0.0.1:53120)
0811 22:18:52.288970 (+ 8924us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:52.288988 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:52.289092 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:52.290635 (+ 1543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:52.290639 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:52.290663 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:52.291679 (+ 1016us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:18:52.291682 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:52.291697 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:18:52.291745 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:52.292256 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:52.292267 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:52.292277 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 22:18:52.292589 (+ 312us) client_negotiation.cc:240] Negotiation successful
0811 22:18:52.292691 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8790,"mutex_wait_us":44,"thread_start_us":8179,"threads_started":1}
I20260811 22:18:52.297223 7096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59535e0cd069456da72ecdc23e7ab316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:52.300758 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 59535e0cd069456da72ecdc23e7ab316 (DEFAULT_TABLE table=test-table [id=1c46333fd2e24e51ba161b6e14ba9c40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:18:52.302789 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59535e0cd069456da72ecdc23e7ab316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:52.309253 7477 tablet_bootstrap.cc:492] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c: Bootstrap starting.
I20260811 22:18:52.312021 7478 tablet_bootstrap.cc:492] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce: Bootstrap starting.
I20260811 22:18:52.314608 7477 tablet_bootstrap.cc:654] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:52.315771 7478 tablet_bootstrap.cc:654] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:52.315987 7477 log.cc:824] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:52.316890 7481 tablet_bootstrap.cc:492] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e: Bootstrap starting.
I20260811 22:18:52.317162 7478 log.cc:824] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:52.320523 7478 tablet_bootstrap.cc:492] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce: No bootstrap required, opened a new log
I20260811 22:18:52.320616 7481 tablet_bootstrap.cc:654] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:52.320825 7478 ts_tablet_manager.cc:1397] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260811 22:18:52.321647 7481 log.cc:824] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:52.321904 7477 tablet_bootstrap.cc:492] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c: No bootstrap required, opened a new log
I20260811 22:18:52.322259 7477 ts_tablet_manager.cc:1397] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260811 22:18:52.324393 7481 tablet_bootstrap.cc:492] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e: No bootstrap required, opened a new log
I20260811 22:18:52.324697 7481 ts_tablet_manager.cc:1397] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.006s
I20260811 22:18:52.332258 7478 raft_consensus.cc:348] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75c8b6c8be9464c822fd757cda5e6ce" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36391 } } peers { permanent_uuid: "979deba5352b4765bb4081a8c91df29c" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45337 } } peers { permanent_uuid: "e8f02d604d4c4d73ae2b9d64e9d3693e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 42629 } }
I20260811 22:18:52.332846 7478 raft_consensus.cc:374] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:52.333030 7478 raft_consensus.cc:729] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75c8b6c8be9464c822fd757cda5e6ce, State: Initialized, Role: FOLLOWER
I20260811 22:18:52.333197 7477 raft_consensus.cc:348] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75c8b6c8be9464c822fd757cda5e6ce" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36391 } } peers { permanent_uuid: "979deba5352b4765bb4081a8c91df29c" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45337 } } peers { permanent_uuid: "e8f02d604d4c4d73ae2b9d64e9d3693e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 42629 } }
I20260811 22:18:52.333215 7481 raft_consensus.cc:348] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75c8b6c8be9464c822fd757cda5e6ce" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36391 } } peers { permanent_uuid: "979deba5352b4765bb4081a8c91df29c" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45337 } } peers { permanent_uuid: "e8f02d604d4c4d73ae2b9d64e9d3693e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 42629 } }
I20260811 22:18:52.333642 7477 raft_consensus.cc:374] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:52.333685 7481 raft_consensus.cc:374] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:52.333745 7477 raft_consensus.cc:729] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 979deba5352b4765bb4081a8c91df29c, State: Initialized, Role: FOLLOWER
I20260811 22:18:52.333791 7481 raft_consensus.cc:729] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8f02d604d4c4d73ae2b9d64e9d3693e, State: Initialized, Role: FOLLOWER
I20260811 22:18:52.333765 7478 consensus_queue.cc:260] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75c8b6c8be9464c822fd757cda5e6ce" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36391 } } peers { permanent_uuid: "979deba5352b4765bb4081a8c91df29c" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45337 } } peers { permanent_uuid: "e8f02d604d4c4d73ae2b9d64e9d3693e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 42629 } }
I20260811 22:18:52.334347 7477 consensus_queue.cc:260] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75c8b6c8be9464c822fd757cda5e6ce" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36391 } } peers { permanent_uuid: "979deba5352b4765bb4081a8c91df29c" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45337 } } peers { permanent_uuid: "e8f02d604d4c4d73ae2b9d64e9d3693e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 42629 } }
I20260811 22:18:52.334507 7481 consensus_queue.cc:260] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75c8b6c8be9464c822fd757cda5e6ce" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36391 } } peers { permanent_uuid: "979deba5352b4765bb4081a8c91df29c" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45337 } } peers { permanent_uuid: "e8f02d604d4c4d73ae2b9d64e9d3693e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 42629 } }
I20260811 22:18:52.335134 7218 heartbeater.cc:503] Master 127.29.229.62:41453 was elected leader, sending a full tablet report...
I20260811 22:18:52.336133 7478 ts_tablet_manager.cc:1428] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260811 22:18:52.337260 7449 heartbeater.cc:503] Master 127.29.229.62:41453 was elected leader, sending a full tablet report...
I20260811 22:18:52.338413 7481 ts_tablet_manager.cc:1428] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e: Time spent starting tablet: real 0.014s user 0.009s sys 0.002s
I20260811 22:18:52.338505 7477 ts_tablet_manager.cc:1428] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c: Time spent starting tablet: real 0.016s user 0.011s sys 0.003s
I20260811 22:18:52.359902 7487 raft_consensus.cc:482] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:18:52.360421 7487 raft_consensus.cc:504] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75c8b6c8be9464c822fd757cda5e6ce" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36391 } } peers { permanent_uuid: "979deba5352b4765bb4081a8c91df29c" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45337 } } peers { permanent_uuid: "e8f02d604d4c4d73ae2b9d64e9d3693e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 42629 } }
I20260811 22:18:52.362475 7487 leader_election.cc:288] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75c8b6c8be9464c822fd757cda5e6ce (127.29.229.2:36391), 979deba5352b4765bb4081a8c91df29c (127.29.229.3:45337)
I20260811 22:18:52.366276 7470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.363140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:60681 (local address 127.29.229.3:45337)
0811 22:18:52.363349 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:52.363355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:52.363947 (+ 592us) server_negotiation.cc:407] Connection header received
0811 22:18:52.364198 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:52.364208 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:52.364259 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:52.364366 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:52.365610 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:52.365614 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:52.365618 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:18:52.365636 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:52.365668 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:18:52.365680 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:52.365789 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:52.365929 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 22:18:52.365996 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 22:18:52.366431 7489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.362776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:45337 (local address 127.29.229.1:60681)
0811 22:18:52.363829 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:52.363845 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:52.364014 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:52.364402 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:52.364409 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:52.364450 (+ 41us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:52.365279 (+ 829us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:18:52.365285 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:52.365305 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:18:52.365365 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:52.365775 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:52.365779 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:52.365783 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 22:18:52.365916 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 22:18:52.365961 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":929,"thread_start_us":1138,"threads_started":1}
I20260811 22:18:52.367151 7467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.363205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:36005 (local address 127.29.229.2:36391)
0811 22:18:52.363378 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:52.363384 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:52.364529 (+ 1145us) server_negotiation.cc:407] Connection header received
0811 22:18:52.364716 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:52.364722 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:52.364781 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:52.364900 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:52.366185 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:52.366190 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:52.366194 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:18:52.366212 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:52.366253 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:18:52.366269 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:52.366401 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:52.366811 (+ 410us) server_negotiation.cc:299] Negotiation successful
0811 22:18:52.366874 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:18:52.367226 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.362892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:36391 (local address 127.29.229.1:36005)
0811 22:18:52.364353 (+ 1461us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:52.364376 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:52.364561 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:52.364918 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:52.364923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:52.364948 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:18:52.365954 (+ 1006us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:18:52.365959 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:18:52.365975 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:18:52.366023 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:18:52.366667 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:18:52.366671 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:18:52.366675 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 22:18:52.366796 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 22:18:52.366835 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1334,"thread_start_us":637,"threads_started":1}
I20260811 22:18:52.367568 7352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59535e0cd069456da72ecdc23e7ab316" candidate_uuid: "e8f02d604d4c4d73ae2b9d64e9d3693e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979deba5352b4765bb4081a8c91df29c" is_pre_election: true
I20260811 22:18:52.368216 7352 raft_consensus.cc:2454] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8f02d604d4c4d73ae2b9d64e9d3693e in term 0.
I20260811 22:18:52.368196 7132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59535e0cd069456da72ecdc23e7ab316" candidate_uuid: "e8f02d604d4c4d73ae2b9d64e9d3693e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75c8b6c8be9464c822fd757cda5e6ce" is_pre_election: true
I20260811 22:18:52.368767 7132 raft_consensus.cc:2454] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8f02d604d4c4d73ae2b9d64e9d3693e in term 0.
I20260811 22:18:52.369043 6775 leader_election.cc:302] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 979deba5352b4765bb4081a8c91df29c, e8f02d604d4c4d73ae2b9d64e9d3693e; no voters:
I20260811 22:18:52.369495 7487 raft_consensus.cc:2781] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:18:52.369655 7487 raft_consensus.cc:482] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:18:52.369764 7487 raft_consensus.cc:3037] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:52.373286 7487 raft_consensus.cc:504] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75c8b6c8be9464c822fd757cda5e6ce" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36391 } } peers { permanent_uuid: "979deba5352b4765bb4081a8c91df29c" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45337 } } peers { permanent_uuid: "e8f02d604d4c4d73ae2b9d64e9d3693e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 42629 } }
I20260811 22:18:52.374018 7487 leader_election.cc:288] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [CANDIDATE]: Term 1 election: Requested vote from peers c75c8b6c8be9464c822fd757cda5e6ce (127.29.229.2:36391), 979deba5352b4765bb4081a8c91df29c (127.29.229.3:45337)
I20260811 22:18:52.374498 7132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59535e0cd069456da72ecdc23e7ab316" candidate_uuid: "e8f02d604d4c4d73ae2b9d64e9d3693e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75c8b6c8be9464c822fd757cda5e6ce"
I20260811 22:18:52.374554 7352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59535e0cd069456da72ecdc23e7ab316" candidate_uuid: "e8f02d604d4c4d73ae2b9d64e9d3693e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979deba5352b4765bb4081a8c91df29c"
I20260811 22:18:52.374774 7352 raft_consensus.cc:3037] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:52.374774 7132 raft_consensus.cc:3037] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:52.378468 7352 raft_consensus.cc:2454] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8f02d604d4c4d73ae2b9d64e9d3693e in term 1.
I20260811 22:18:52.378468 7132 raft_consensus.cc:2454] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8f02d604d4c4d73ae2b9d64e9d3693e in term 1.
I20260811 22:18:52.378978 6779 leader_election.cc:302] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c75c8b6c8be9464c822fd757cda5e6ce, e8f02d604d4c4d73ae2b9d64e9d3693e; no voters:
I20260811 22:18:52.379395 7487 raft_consensus.cc:2781] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:52.379987 7487 raft_consensus.cc:686] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [term 1 LEADER]: Becoming Leader. State: Replica: e8f02d604d4c4d73ae2b9d64e9d3693e, State: Running, Role: LEADER
I20260811 22:18:52.380620 7487 consensus_queue.cc:237] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75c8b6c8be9464c822fd757cda5e6ce" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36391 } } peers { permanent_uuid: "979deba5352b4765bb4081a8c91df29c" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45337 } } peers { permanent_uuid: "e8f02d604d4c4d73ae2b9d64e9d3693e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 42629 } }
I20260811 22:18:52.386201 6640 catalog_manager.cc:5495] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e reported cstate change: term changed from 0 to 1, leader changed from <none> to e8f02d604d4c4d73ae2b9d64e9d3693e (127.29.229.1). New cstate: current_term: 1 leader_uuid: "e8f02d604d4c4d73ae2b9d64e9d3693e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75c8b6c8be9464c822fd757cda5e6ce" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "979deba5352b4765bb4081a8c91df29c" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8f02d604d4c4d73ae2b9d64e9d3693e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 42629 } health_report { overall_health: HEALTHY } } }
I20260811 22:18:52.414466 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.412250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53134 (local address 127.29.229.1:42629)
0811 22:18:52.412454 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:52.412460 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:52.412506 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:18:52.412707 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:52.412711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:52.412807 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:52.412932 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:52.413831 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:52.413835 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:52.413852 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:18:52.413867 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:52.413897 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:18:52.413909 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:52.414042 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:52.414146 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 22:18:52.414207 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:18:52.423175 7132 raft_consensus.cc:1261] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce [term 1 FOLLOWER]: Refusing update from remote peer e8f02d604d4c4d73ae2b9d64e9d3693e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:52.423204 7352 raft_consensus.cc:1261] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c [term 1 FOLLOWER]: Refusing update from remote peer e8f02d604d4c4d73ae2b9d64e9d3693e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:18:52.424547 7487 consensus_queue.cc:1035] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [LEADER]: Connected to new peer: Peer: permanent_uuid: "979deba5352b4765bb4081a8c91df29c" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:52.424909 7491 consensus_queue.cc:1035] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75c8b6c8be9464c822fd757cda5e6ce" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:18:52.440507 7496 mvcc.cc:205] Tried to move back new op lower bound from 7317449655993712640 to 7317449655835316224. Current Snapshot: MvccSnapshot[applied={T|T < 7317449655993712640}]
I20260811 22:18:52.456858 6639 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:18:52.457202 6639 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:18:52.461205 6639 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a9a5b91174b41ec918839db1628c170: Sending DeleteTablet for 3 replicas of tablet 59535e0cd069456da72ecdc23e7ab316
I20260811 22:18:52.462561 7096 tablet_service.cc:1514] Processing DeleteTablet for tablet 59535e0cd069456da72ecdc23e7ab316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:52 UTC) from {username='slave'} at 127.0.0.1:47272
I20260811 22:18:52.462636 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 59535e0cd069456da72ecdc23e7ab316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:52 UTC) from {username='slave'} at 127.0.0.1:53120
I20260811 22:18:52.463379 7325 tablet_service.cc:1514] Processing DeleteTablet for tablet 59535e0cd069456da72ecdc23e7ab316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:18:52 UTC) from {username='slave'} at 127.0.0.1:48170
I20260811 22:18:52.464035 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 6748
I20260811 22:18:52.477375 7513 tablet_replica.cc:330] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c: stopping tablet replica
I20260811 22:18:52.478006 7513 raft_consensus.cc:2229] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:18:52.479041 7513 raft_consensus.cc:2258] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:52.479823 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.465319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53144 (local address 127.29.229.1:42629)
0811 22:18:52.465511 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:52.465517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:52.476994 (+ 11477us) server_negotiation.cc:407] Connection header received
0811 22:18:52.477353 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:52.477360 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:52.477430 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:52.477515 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:52.478720 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:52.478725 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:52.478730 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:18:52.478750 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:52.478785 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:18:52.478801 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:52.478930 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:52.479378 (+ 448us) server_negotiation.cc:299] Negotiation successful
0811 22:18:52.479458 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:18:52.479911 7512 tablet_replica.cc:330] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e: stopping tablet replica
I20260811 22:18:52.480319 7512 raft_consensus.cc:2229] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:18:52.482420 7513 ts_tablet_manager.cc:1905] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:52.481137 6975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:52.488066 7513 ts_tablet_manager.cc:1918] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:18:52.488272 7513 log.cc:1196] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/59535e0cd069456da72ecdc23e7ab316
I20260811 22:18:52.489002 7513 ts_tablet_manager.cc:1939] T 59535e0cd069456da72ecdc23e7ab316 P 979deba5352b4765bb4081a8c91df29c: Deleting consensus metadata
I20260811 22:18:52.489410 7511 tablet_replica.cc:330] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce: stopping tablet replica
I20260811 22:18:52.489913 7511 raft_consensus.cc:2229] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:18:52.490626 7511 raft_consensus.cc:2258] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:52.491328 6610 catalog_manager.cc:4841] TS 979deba5352b4765bb4081a8c91df29c (127.29.229.3:45337): tablet 59535e0cd069456da72ecdc23e7ab316 (table test-table [id=1c46333fd2e24e51ba161b6e14ba9c40]) successfully deleted
I20260811 22:18:52.493302 7511 ts_tablet_manager.cc:1905] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:52.500659 7511 ts_tablet_manager.cc:1918] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:18:52.500873 7511 log.cc:1196] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/59535e0cd069456da72ecdc23e7ab316
I20260811 22:18:52.501421 7511 ts_tablet_manager.cc:1939] T 59535e0cd069456da72ecdc23e7ab316 P c75c8b6c8be9464c822fd757cda5e6ce: Deleting consensus metadata
I20260811 22:18:52.503494 6612 catalog_manager.cc:4841] TS c75c8b6c8be9464c822fd757cda5e6ce (127.29.229.2:36391): tablet 59535e0cd069456da72ecdc23e7ab316 (table test-table [id=1c46333fd2e24e51ba161b6e14ba9c40]) successfully deleted
I20260811 22:18:52.588388 7512 raft_consensus.cc:2258] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:18:52.590758 7512 ts_tablet_manager.cc:1905] T 59535e0cd069456da72ecdc23e7ab316 P e8f02d604d4c4d73ae2b9d64e9d3693e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:18:52.593542 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 6748
W20260811 22:18:52.622025 6611 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.229.1:42629 (error 108)
I20260811 22:18:52.622236 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 6992
I20260811 22:18:52.624220 7469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.623689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:42629 (local address 127.0.0.1:53158)
0811 22:18:52.623878 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:52.624042 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 22:18:52.624509 6611 catalog_manager.cc:4584] TS e8f02d604d4c4d73ae2b9d64e9d3693e (127.29.229.1:42629): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 59535e0cd069456da72ecdc23e7ab316: Network error: Client connection negotiation failed: client connection to 127.29.229.1:42629: connect: Connection refused (error 111)
I20260811 22:18:52.625519 7467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.623138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47280 (local address 127.29.229.2:36391)
0811 22:18:52.623308 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:52.623316 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:52.623428 (+ 112us) server_negotiation.cc:407] Connection header received
0811 22:18:52.623616 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:52.623623 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:52.623748 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:52.623870 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:52.624875 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:52.624883 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:52.624890 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:18:52.624908 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:52.624946 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:18:52.624962 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:52.625087 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:52.625187 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 22:18:52.625249 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:18:52.626264 7207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:52.678675 7469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.678109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:42629 (local address 127.0.0.1:53174)
0811 22:18:52.678359 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:52.678473 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 22:18:52.679891 7469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.679494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:42629 (local address 127.0.0.1:53182)
0811 22:18:52.679650 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:52.679748 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 22:18:52.719507 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 6992
I20260811 22:18:52.740749 7469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.740034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:42629 (local address 127.0.0.1:53194)
0811 22:18:52.740356 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:52.740520 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 22:18:52.742256 7469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.741759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:42629 (local address 127.0.0.1:53200)
0811 22:18:52.741978 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:52.742082 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 22:18:52.751187 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 7222
I20260811 22:18:52.754650 7470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.752043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48186 (local address 127.29.229.3:45337)
0811 22:18:52.752435 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:52.752441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:52.752473 (+ 32us) server_negotiation.cc:407] Connection header received
0811 22:18:52.752620 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:52.752624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:52.752738 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:52.752880 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:52.753871 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:52.753877 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:52.753881 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:18:52.753903 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:52.753951 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:18:52.753966 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:52.754103 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:52.754237 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 22:18:52.754299 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260811 22:18:52.755420 7439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:52.823143 7469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.822591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:42629 (local address 127.0.0.1:53208)
0811 22:18:52.822840 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:52.822961 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 22:18:52.824458 7469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:52.823974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:42629 (local address 127.0.0.1:53220)
0811 22:18:52.824224 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:52.824309 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260811 22:18:52.848766 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 7222
I20260811 22:18:52.875267 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 6578
I20260811 22:18:52.876763 6705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:52.990859 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 6578
2026-08-11T22:18:53Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2875 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 22:18:53.052906 30612 mini_oidc.cc:152] Starting JWKS server
I20260811 22:18:53.056627 30612 webserver.cc:460] Webserver started at https://127.0.0.1:37385/ using document root <none> and password file <none>
I20260811 22:18:53.057068 30612 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:18:53.057806 30612 webserver.cc:460] Webserver started at http://127.0.0.1:39233/ using document root <none> and password file <none>
2026-08-11T22:18:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:18:53Z Disabled control of system clock
I20260811 22:18:53.100245 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:38167
--webserver_interface=127.29.229.62
--webserver_port=0
--builtin_ntp_servers=127.29.229.20:36831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:38167
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37385/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0/testchainca.cert with env {}
W20260811 22:18:53.468585 7535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:53.468962 7535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:53.469127 7535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:53.476431 7535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:53.476615 7535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:53.476715 7535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:53.476831 7535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:53.476925 7535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:18:53.477022 7535 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:18:53.477102 7535 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37385/jwks.json
I20260811 22:18:53.487277 7535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:36831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:38167
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:38167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37385/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:53.488758 7535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:53.490175 7535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:53.502748 7544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:53.502802 7540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:53.502838 7541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:53.504070 7535 server_base.cc:1034] running on GCE node
I20260811 22:18:53.505199 7535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:53.511960 7535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:53.513288 7535 hybrid_clock.cc:648] HybridClock initialized: now 1786486733513179 us; error 107 us; skew 500 ppm
I20260811 22:18:53.513859 7535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:18:53.516670 7535 webserver.cc:460] Webserver started at http://127.29.229.62:44563/ using document root <none> and password file <none>
I20260811 22:18:53.517459 7535 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:53.517562 7535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:53.517928 7535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:53.521497 7535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "1ec93af466a8492e96a95c9d1b743626"
format_stamp: "Formatted at 2026-08-11 22:18:53 on dist-test-slave-jq4t"
server_key: "3ad19afe0611dd0019a43b81bfdd9379"
server_key_iv: "4786c1c24fea82283be1fb8015bed4e1"
server_key_version: "encryptionkey@0"
I20260811 22:18:53.522411 7535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "1ec93af466a8492e96a95c9d1b743626"
format_stamp: "Formatted at 2026-08-11 22:18:53 on dist-test-slave-jq4t"
server_key: "3ad19afe0611dd0019a43b81bfdd9379"
server_key_iv: "4786c1c24fea82283be1fb8015bed4e1"
server_key_version: "encryptionkey@0"
I20260811 22:18:53.528029 7535 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 22:18:53.532397 7551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:53.534384 7535 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:18:53.534789 7535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "1ec93af466a8492e96a95c9d1b743626"
format_stamp: "Formatted at 2026-08-11 22:18:53 on dist-test-slave-jq4t"
server_key: "3ad19afe0611dd0019a43b81bfdd9379"
server_key_iv: "4786c1c24fea82283be1fb8015bed4e1"
server_key_version: "encryptionkey@0"
I20260811 22:18:53.535322 7535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:53.566099 7535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:53.567003 7535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:53.567334 7535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:53.586709 7535 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:38167
I20260811 22:18:53.586737 7642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:38167 every 8 connection(s)
I20260811 22:18:53.588138 7535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:53.592093 7643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:53.593681 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 7535
I20260811 22:18:53.593946 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:18:53.594585 30612 external_mini_cluster.cc:1428] Setting key 10fbb0d42c3bf72a338e11ab95f7b953
I20260811 22:18:53.604022 7643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626: Bootstrap starting.
I20260811 22:18:53.607348 7643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:53.608549 7643 log.cc:824] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:53.610095 7649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:53.596578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57998 (local address 127.29.229.62:38167)
0811 22:18:53.597803 (+ 1225us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:53.597818 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:53.597862 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:18:53.599601 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:53.599622 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:53.600064 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:53.601068 (+ 1004us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:53.601909 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:53.603207 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:53.604941 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:53.605674 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:53.605912 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:53.607165 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:53.607172 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:53.607187 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:18:53.607816 (+ 629us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:53.607917 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:18:53.607939 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:53.608078 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:53.608316 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 22:18:53.608697 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":438,"threads_started":1}
I20260811 22:18:53.612215 7643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626: No bootstrap required, opened a new log
I20260811 22:18:53.614279 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:18:53.621152 7643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec93af466a8492e96a95c9d1b743626" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 38167 } }
I20260811 22:18:53.621625 7643 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:53.621726 7643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ec93af466a8492e96a95c9d1b743626, State: Initialized, Role: FOLLOWER
I20260811 22:18:53.622301 7643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec93af466a8492e96a95c9d1b743626" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 38167 } }
I20260811 22:18:53.622542 7643 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:18:53.622680 7643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:18:53.622855 7643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:53.625180 7643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec93af466a8492e96a95c9d1b743626" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 38167 } }
I20260811 22:18:53.625720 7643 leader_election.cc:302] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ec93af466a8492e96a95c9d1b743626; no voters:
I20260811 22:18:53.626327 7643 leader_election.cc:288] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:18:53.626536 7655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:53.627362 7655 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626 [term 1 LEADER]: Becoming Leader. State: Replica: 1ec93af466a8492e96a95c9d1b743626, State: Running, Role: LEADER
I20260811 22:18:53.627964 7655 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec93af466a8492e96a95c9d1b743626" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 38167 } }
I20260811 22:18:53.628780 7643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:53.631008 7657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ec93af466a8492e96a95c9d1b743626. Latest consensus state: current_term: 1 leader_uuid: "1ec93af466a8492e96a95c9d1b743626" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec93af466a8492e96a95c9d1b743626" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 38167 } } }
I20260811 22:18:53.631453 7657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:53.630982 7656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ec93af466a8492e96a95c9d1b743626" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec93af466a8492e96a95c9d1b743626" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 38167 } } }
I20260811 22:18:53.631657 7656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:53.641736 7660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:53.647177 7660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:53.659853 7660 catalog_manager.cc:1347] Generated new cluster ID: 632b2e6ca5464d4e896589708025084d
I20260811 22:18:53.659988 7660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:53.715428 7660 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:18:53.716878 7660 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:53.735293 7660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ec93af466a8492e96a95c9d1b743626: Generated new TSK 0
I20260811 22:18:53.736290 7660 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 22:18:53.788220 7702 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 22:18:53.788530 7702 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 22:18:53.790725 7649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:53.757252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58014 (local address 127.29.229.62:38167)
0811 22:18:53.757525 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:53.757531 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:53.758264 (+ 733us) server_negotiation.cc:407] Connection header received
0811 22:18:53.758781 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:53.758786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:53.758891 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:53.759038 (+ 147us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:18:53.759756 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:53.760740 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:53.762935 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:53.763389 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:53.763500 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:53.764173 (+ 673us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:18:53.790048 (+ 25875us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37385/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 22:18:53.790478 (+ 430us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58014: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37385/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":82}
W20260811 22:18:53.790946 7649 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58014: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37385/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 22:18:53.794005 7699 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:18:53.757147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:38167 (local address 127.0.0.1:58014)
0811 22:18:53.757963 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:53.758040 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:53.758404 (+ 364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:53.759099 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:53.759107 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:53.759124 (+ 17us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:18:53.759562 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:53.759576 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:53.760903 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:53.760910 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:53.762729 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:53.762744 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:53.763605 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:53.763614 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:53.763784 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:53.763875 (+ 91us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:18:53.790523 (+ 26648us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37385/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 22:18:53.791087 (+ 564us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.229.62:38167: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37385/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":338,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.29.229.62:38167: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37385/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 22:18:53.796569 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 7535
I20260811 22:18:53.798007 7635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:53.904393 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 7535
2026-08-11T22:18:53Z chronyd exiting
I20260811 22:18:53.968319 30612 test_util.cc:175] -----------------------------------------------
I20260811 22:18:53.968544 30612 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486685119861-30612-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (920 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 22:18:53.973556 30612 mini_oidc.cc:152] Starting JWKS server
I20260811 22:18:53.975479 30612 webserver.cc:460] Webserver started at https://127.0.0.1:33485/ using document root <none> and password file <none>
I20260811 22:18:53.975741 30612 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:18:53.976521 30612 webserver.cc:460] Webserver started at http://127.0.0.1:39913/ using document root <none> and password file <none>
2026-08-11T22:18:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:18:53Z Disabled control of system clock
I20260811 22:18:54.019400 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:41283
--webserver_interface=127.29.229.62
--webserver_port=0
--builtin_ntp_servers=127.29.229.20:36129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:41283
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33485/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486685119861-30612-0/testchainca.cert with env {}
W20260811 22:18:54.372815 7717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:54.373108 7717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:54.373301 7717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:54.382161 7717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:54.382266 7717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:54.382362 7717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:54.382467 7717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:54.382529 7717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:18:54.382623 7717 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:18:54.382694 7717 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33485/jwks.json
I20260811 22:18:54.392997 7717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:36129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:41283
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:41283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33485/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486685119861-30612-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:54.394335 7717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:54.395767 7717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:54.407994 7724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:54.407953 7730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:54.408032 7725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:54.408818 7717 server_base.cc:1034] running on GCE node
I20260811 22:18:54.409641 7717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:54.411019 7717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:54.412247 7717 hybrid_clock.cc:648] HybridClock initialized: now 1786486734412217 us; error 48 us; skew 500 ppm
I20260811 22:18:54.412655 7717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:18:54.414952 7717 webserver.cc:460] Webserver started at http://127.29.229.62:45455/ using document root <none> and password file <none>
I20260811 22:18:54.415647 7717 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:54.415784 7717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:54.416210 7717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:54.419127 7717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "3959cffd2b1e4049b78cb443e59daf55"
format_stamp: "Formatted at 2026-08-11 22:18:54 on dist-test-slave-jq4t"
server_key: "3fe5b5114b9601ee9a8f8149adedcac8"
server_key_iv: "3620ed3cff6522ed1f3ef33e227de05e"
server_key_version: "encryptionkey@0"
I20260811 22:18:54.419965 7717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "3959cffd2b1e4049b78cb443e59daf55"
format_stamp: "Formatted at 2026-08-11 22:18:54 on dist-test-slave-jq4t"
server_key: "3fe5b5114b9601ee9a8f8149adedcac8"
server_key_iv: "3620ed3cff6522ed1f3ef33e227de05e"
server_key_version: "encryptionkey@0"
I20260811 22:18:54.425251 7717 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 22:18:54.428944 7739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:54.430300 7717 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:18:54.430516 7717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "3959cffd2b1e4049b78cb443e59daf55"
format_stamp: "Formatted at 2026-08-11 22:18:54 on dist-test-slave-jq4t"
server_key: "3fe5b5114b9601ee9a8f8149adedcac8"
server_key_iv: "3620ed3cff6522ed1f3ef33e227de05e"
server_key_version: "encryptionkey@0"
I20260811 22:18:54.430747 7717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:54.472137 7717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:54.473096 7717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:54.473443 7717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:54.494619 7717 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:41283
I20260811 22:18:54.494637 7839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:41283 every 8 connection(s)
I20260811 22:18:54.496091 7717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:54.498176 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 7717
I20260811 22:18:54.498386 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:18:54.499259 30612 external_mini_cluster.cc:1428] Setting key 15cf9f3b61bc2bc4b0a5ab6387c7e0e2
I20260811 22:18:54.500973 7841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:54.514436 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.501552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36168 (local address 127.29.229.62:41283)
0811 22:18:54.502600 (+ 1048us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:54.502611 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:54.502653 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:18:54.504251 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:54.504271 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:54.504793 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:54.505120 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:54.505697 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.507078 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.508917 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.509594 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.509733 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.511263 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:54.511270 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:54.511286 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:18:54.511889 (+ 603us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:54.511980 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:18:54.512002 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:54.512103 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:54.512401 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 22:18:54.512800 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":411,"threads_started":1}
I20260811 22:18:54.516288 7841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55: Bootstrap starting.
I20260811 22:18:54.520241 7841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:54.520422 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:18:54.521449 7841 log.cc:824] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:54.524631 7841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55: No bootstrap required, opened a new log
I20260811 22:18:54.532618 7841 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3959cffd2b1e4049b78cb443e59daf55" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41283 } }
I20260811 22:18:54.533048 7841 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:54.533179 7841 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3959cffd2b1e4049b78cb443e59daf55, State: Initialized, Role: FOLLOWER
I20260811 22:18:54.533783 7841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3959cffd2b1e4049b78cb443e59daf55" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41283 } }
I20260811 22:18:54.534060 7841 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:18:54.534183 7841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:18:54.534368 7841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:54.537539 7841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3959cffd2b1e4049b78cb443e59daf55" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41283 } }
I20260811 22:18:54.538192 7841 leader_election.cc:302] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3959cffd2b1e4049b78cb443e59daf55; no voters:
I20260811 22:18:54.538827 7841 leader_election.cc:288] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:18:54.539002 7849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:54.539774 7849 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55 [term 1 LEADER]: Becoming Leader. State: Replica: 3959cffd2b1e4049b78cb443e59daf55, State: Running, Role: LEADER
I20260811 22:18:54.540391 7849 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3959cffd2b1e4049b78cb443e59daf55" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41283 } }
I20260811 22:18:54.541224 7841 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:54.543653 7850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3959cffd2b1e4049b78cb443e59daf55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3959cffd2b1e4049b78cb443e59daf55" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41283 } } }
I20260811 22:18:54.544045 7850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:54.544543 7855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3959cffd2b1e4049b78cb443e59daf55. Latest consensus state: current_term: 1 leader_uuid: "3959cffd2b1e4049b78cb443e59daf55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3959cffd2b1e4049b78cb443e59daf55" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41283 } } }
I20260811 22:18:54.544873 7855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:54.545708 7865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:54.552023 7865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:54.561434 7865 catalog_manager.cc:1347] Generated new cluster ID: f0342a3a20e3437dada7e4e0bd1a3180
I20260811 22:18:54.561583 7865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:54.571158 7865 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:18:54.572467 7865 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:54.582235 7865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3959cffd2b1e4049b78cb443e59daf55: Generated new TSK 0
I20260811 22:18:54.583143 7865 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 22:18:54.623608 7909 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 22:18:54.623785 7909 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 22:18:54.625120 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:54.602537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36184 (local address 127.29.229.62:41283)
0811 22:18:54.602797 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:54.602805 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:54.603461 (+ 656us) server_negotiation.cc:407] Connection header received
0811 22:18:54.603711 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:54.603717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:54.603818 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:54.604011 (+ 193us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:18:54.604553 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.605728 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.607233 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.607942 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.608083 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.608705 (+ 622us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:18:54.624307 (+ 15602us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 22:18:54.624774 (+ 467us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36184: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":95}
W20260811 22:18:54.625258 7846 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36184: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 22:18:54.625774 7907 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:18:54.602341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41283 (local address 127.0.0.1:36184)
0811 22:18:54.603341 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:54.603363 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:54.603508 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:54.604063 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:54.604067 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:54.604078 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:18:54.604357 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.604365 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.605916 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.605920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.607079 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:54.607091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:54.608169 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:54.608174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:54.608328 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:54.608365 (+ 37us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:18:54.624685 (+ 16320us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 22:18:54.625030 (+ 345us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.229.62:41283: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":526,"threads_started":1}
I20260811 22:18:54.627231 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 7717
I20260811 22:18:54.628756 7834 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:54.726646 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 7717
2026-08-11T22:18:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (821 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 22:18:54.793790 30612 mini_oidc.cc:152] Starting JWKS server
I20260811 22:18:54.795774 30612 webserver.cc:460] Webserver started at https://127.0.0.1:39623/ using document root <none> and password file <none>
I20260811 22:18:54.796043 30612 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:18:54.796754 30612 webserver.cc:460] Webserver started at http://127.0.0.1:43047/ using document root <none> and password file <none>
2026-08-11T22:18:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:18:54Z Disabled control of system clock
I20260811 22:18:54.838572 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:37499
--webserver_interface=127.29.229.62
--webserver_port=0
--builtin_ntp_servers=127.29.229.20:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:37499
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39623/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 22:18:55.221503 7918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:55.221880 7918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:55.222048 7918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:55.229597 7918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:55.229765 7918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:55.229861 7918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:55.229981 7918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:55.230064 7918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:18:55.230158 7918 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:18:55.230239 7918 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39623/jwks.json
I20260811 22:18:55.239629 7918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:37499
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:37499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39623/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:55.241075 7918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:55.242874 7918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:55.256230 7923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:55.256278 7924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:55.256309 7926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:55.257237 7918 server_base.cc:1034] running on GCE node
I20260811 22:18:55.258080 7918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:55.259608 7918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:55.260870 7918 hybrid_clock.cc:648] HybridClock initialized: now 1786486735260801 us; error 67 us; skew 500 ppm
I20260811 22:18:55.261263 7918 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:18:55.263727 7918 webserver.cc:460] Webserver started at http://127.29.229.62:39505/ using document root <none> and password file <none>
I20260811 22:18:55.264559 7918 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:55.264681 7918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:55.265035 7918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:55.267969 7918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "e9dbea9d341a4974861e0ce1a1e342aa"
format_stamp: "Formatted at 2026-08-11 22:18:55 on dist-test-slave-jq4t"
server_key: "cb3a71d2a3622e32c3826c6ca264d67b"
server_key_iv: "ba60683ce5c6378e5ac3b37a92b26b2e"
server_key_version: "encryptionkey@0"
I20260811 22:18:55.268870 7918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "e9dbea9d341a4974861e0ce1a1e342aa"
format_stamp: "Formatted at 2026-08-11 22:18:55 on dist-test-slave-jq4t"
server_key: "cb3a71d2a3622e32c3826c6ca264d67b"
server_key_iv: "ba60683ce5c6378e5ac3b37a92b26b2e"
server_key_version: "encryptionkey@0"
I20260811 22:18:55.274515 7918 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 22:18:55.279026 7937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:55.281267 7918 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 22:18:55.281502 7918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "e9dbea9d341a4974861e0ce1a1e342aa"
format_stamp: "Formatted at 2026-08-11 22:18:55 on dist-test-slave-jq4t"
server_key: "cb3a71d2a3622e32c3826c6ca264d67b"
server_key_iv: "ba60683ce5c6378e5ac3b37a92b26b2e"
server_key_version: "encryptionkey@0"
I20260811 22:18:55.281773 7918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:55.312448 7918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:55.313391 7918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:55.313740 7918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:55.335033 7918 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:37499
I20260811 22:18:55.335052 8021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:37499 every 8 connection(s)
I20260811 22:18:55.337075 7918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:55.341130 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 7918
I20260811 22:18:55.341367 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:18:55.341421 8022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:55.342275 30612 external_mini_cluster.cc:1428] Setting key e1105bf889480418e9a84646884efc51
I20260811 22:18:55.356909 8022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa: Bootstrap starting.
I20260811 22:18:55.358477 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:55.344742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34010 (local address 127.29.229.62:37499)
0811 22:18:55.346334 (+ 1592us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:55.346345 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:55.346392 (+ 47us) server_negotiation.cc:407] Connection header received
0811 22:18:55.348661 (+ 2269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:55.348686 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:55.349182 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:55.349538 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:55.350206 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:55.351785 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:55.353837 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:55.354567 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:55.354718 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:55.356019 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:55.356027 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:55.356041 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:18:55.356666 (+ 625us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:55.356749 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:18:55.356770 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:55.356868 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:55.357181 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 22:18:55.357523 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1050,"thread_start_us":713,"threads_started":1}
I20260811 22:18:55.360267 8022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:55.361578 8022 log.cc:824] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:55.362278 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:18:55.364437 8022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa: No bootstrap required, opened a new log
I20260811 22:18:55.372087 8022 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9dbea9d341a4974861e0ce1a1e342aa" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37499 } }
I20260811 22:18:55.372574 8022 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:55.372673 8022 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9dbea9d341a4974861e0ce1a1e342aa, State: Initialized, Role: FOLLOWER
I20260811 22:18:55.373221 8022 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9dbea9d341a4974861e0ce1a1e342aa" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37499 } }
I20260811 22:18:55.373447 8022 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:18:55.373549 8022 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:18:55.373684 8022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:55.376199 8022 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9dbea9d341a4974861e0ce1a1e342aa" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37499 } }
I20260811 22:18:55.376794 8022 leader_election.cc:302] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9dbea9d341a4974861e0ce1a1e342aa; no voters:
I20260811 22:18:55.377395 8022 leader_election.cc:288] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:18:55.377601 8032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:55.378486 8032 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa [term 1 LEADER]: Becoming Leader. State: Replica: e9dbea9d341a4974861e0ce1a1e342aa, State: Running, Role: LEADER
I20260811 22:18:55.379133 8032 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9dbea9d341a4974861e0ce1a1e342aa" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37499 } }
I20260811 22:18:55.379964 8022 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:55.386509 8033 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9dbea9d341a4974861e0ce1a1e342aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9dbea9d341a4974861e0ce1a1e342aa" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37499 } } }
I20260811 22:18:55.388043 8033 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:55.390623 8034 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9dbea9d341a4974861e0ce1a1e342aa. Latest consensus state: current_term: 1 leader_uuid: "e9dbea9d341a4974861e0ce1a1e342aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9dbea9d341a4974861e0ce1a1e342aa" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37499 } } }
I20260811 22:18:55.392134 8034 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:55.394824 8037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:55.401698 8037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:55.411360 8037 catalog_manager.cc:1347] Generated new cluster ID: 6130a05db5854d8d88bccbcbf3c36643
I20260811 22:18:55.411522 8037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:55.424983 8037 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:18:55.426303 8037 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:55.433017 8037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9dbea9d341a4974861e0ce1a1e342aa: Generated new TSK 0
I20260811 22:18:55.434085 8037 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:18:55.467042 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:55.456879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34016 (local address 127.29.229.62:37499)
0811 22:18:55.457101 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:55.457108 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:55.457590 (+ 482us) server_negotiation.cc:407] Connection header received
0811 22:18:55.457800 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:55.457807 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:55.457921 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:55.458102 (+ 181us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:18:55.458553 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:55.459759 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:55.461294 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:55.462039 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:55.462195 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:55.463394 (+ 1199us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:18:55.466198 (+ 2804us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39623/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 22:18:55.466572 (+ 374us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34016: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39623/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 22:18:55.467252 8072 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:18:55.456781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:37499 (local address 127.0.0.1:34016)
0811 22:18:55.457483 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:55.457499 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:55.457627 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:55.458121 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:55.458125 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:55.458134 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:18:55.458393 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:55.458401 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:55.459928 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:55.459933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:55.461125 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:55.461138 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:55.462446 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:55.462453 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:55.462897 (+ 444us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:55.462993 (+ 96us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:18:55.466473 (+ 3480us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39623/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 22:18:55.466793 (+ 320us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.229.62:37499: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39623/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":384,"threads_started":1}
W20260811 22:18:55.467442 8030 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34016: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39623/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260811 22:18:55.469403 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 7918
I20260811 22:18:55.470847 8016 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 22:18:55.480644 8075 webserver.cc:533] Webserver: sslize failed:
E20260811 22:18:55.480856 8075 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 22:18:55.568706 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 7918
2026-08-11T22:18:55Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (839 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T22:18:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:18:55Z Disabled control of system clock
I20260811 22:18:55.672416 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:38701
--webserver_interface=127.29.229.62
--webserver_port=0
--builtin_ntp_servers=127.29.229.20:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:38701
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486685119861-30612-0/testchainca.cert with env {}
W20260811 22:18:56.028923 8084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:56.029258 8084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:56.029434 8084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:56.037938 8084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:56.038089 8084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:56.038160 8084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:56.038251 8084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:56.038348 8084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:18:56.038461 8084 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:18:56.038509 8084 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 22:18:56.049170 8084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:38701
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:38701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486685119861-30612-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:56.050593 8084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:56.051977 8084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:56.064930 8092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:56.064898 8093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:56.064934 8101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:56.065636 8084 server_base.cc:1034] running on GCE node
I20260811 22:18:56.066498 8084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:56.067862 8084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:56.069077 8084 hybrid_clock.cc:648] HybridClock initialized: now 1786486736069037 us; error 34 us; skew 500 ppm
I20260811 22:18:56.069506 8084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:18:56.075253 8084 webserver.cc:460] Webserver started at http://127.29.229.62:35125/ using document root <none> and password file <none>
I20260811 22:18:56.075961 8084 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:56.076058 8084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:56.076345 8084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:56.079559 8084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "089f88d10e554230a5c5005d3617cacf"
format_stamp: "Formatted at 2026-08-11 22:18:56 on dist-test-slave-jq4t"
server_key: "c84657b2ffac4504ff0b44e71c31ec15"
server_key_iv: "e2161231a89b05d8912731c1b569027f"
server_key_version: "encryptionkey@0"
I20260811 22:18:56.080372 8084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "089f88d10e554230a5c5005d3617cacf"
format_stamp: "Formatted at 2026-08-11 22:18:56 on dist-test-slave-jq4t"
server_key: "c84657b2ffac4504ff0b44e71c31ec15"
server_key_iv: "e2161231a89b05d8912731c1b569027f"
server_key_version: "encryptionkey@0"
I20260811 22:18:56.085759 8084 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 22:18:56.090180 8110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:56.092053 8084 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:18:56.092305 8084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "089f88d10e554230a5c5005d3617cacf"
format_stamp: "Formatted at 2026-08-11 22:18:56 on dist-test-slave-jq4t"
server_key: "c84657b2ffac4504ff0b44e71c31ec15"
server_key_iv: "e2161231a89b05d8912731c1b569027f"
server_key_version: "encryptionkey@0"
I20260811 22:18:56.092592 8084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:56.108299 8084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:56.109277 8084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:56.109594 8084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:56.130738 8084 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:38701
I20260811 22:18:56.130772 8210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:38701 every 8 connection(s)
I20260811 22:18:56.132279 8084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:56.136781 8212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:56.141228 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 8084
I20260811 22:18:56.141443 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:18:56.142329 30612 external_mini_cluster.cc:1428] Setting key e26c7d98d5866f2ed5216ecd361bc63f
I20260811 22:18:56.149649 8212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf: Bootstrap starting.
I20260811 22:18:56.152935 8212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:56.154284 8212 log.cc:824] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:56.158576 8212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf: No bootstrap required, opened a new log
I20260811 22:18:56.158598 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:56.144618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48964 (local address 127.29.229.62:38701)
0811 22:18:56.145711 (+ 1093us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:56.145722 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:56.145764 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:18:56.148266 (+ 2502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:56.148301 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:56.148849 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:56.149256 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:56.149987 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:56.151358 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:56.152742 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:56.153512 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:56.153771 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:56.155497 (+ 1726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:56.155513 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:56.155543 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:18:56.156592 (+ 1049us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:56.156706 (+ 114us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:18:56.156734 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:56.156840 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:56.157282 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 22:18:56.157624 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":499,"threads_started":1}
I20260811 22:18:56.163877 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:18:56.167996 8212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089f88d10e554230a5c5005d3617cacf" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 38701 } }
I20260811 22:18:56.168418 8212 raft_consensus.cc:374] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:56.168586 8212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089f88d10e554230a5c5005d3617cacf, State: Initialized, Role: FOLLOWER
I20260811 22:18:56.169291 8212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089f88d10e554230a5c5005d3617cacf" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 38701 } }
I20260811 22:18:56.169567 8212 raft_consensus.cc:388] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:18:56.169723 8212 raft_consensus.cc:482] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:18:56.169922 8212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:56.172165 8212 raft_consensus.cc:504] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089f88d10e554230a5c5005d3617cacf" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 38701 } }
I20260811 22:18:56.172775 8212 leader_election.cc:302] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 089f88d10e554230a5c5005d3617cacf; no voters:
I20260811 22:18:56.173449 8212 leader_election.cc:288] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:18:56.173628 8218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:56.174750 8218 raft_consensus.cc:686] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf [term 1 LEADER]: Becoming Leader. State: Replica: 089f88d10e554230a5c5005d3617cacf, State: Running, Role: LEADER
I20260811 22:18:56.175328 8218 consensus_queue.cc:237] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089f88d10e554230a5c5005d3617cacf" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 38701 } }
I20260811 22:18:56.175719 8212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:56.178944 8219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "089f88d10e554230a5c5005d3617cacf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089f88d10e554230a5c5005d3617cacf" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 38701 } } }
I20260811 22:18:56.179960 8219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:56.179005 8222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 089f88d10e554230a5c5005d3617cacf. Latest consensus state: current_term: 1 leader_uuid: "089f88d10e554230a5c5005d3617cacf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089f88d10e554230a5c5005d3617cacf" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 38701 } } }
I20260811 22:18:56.183463 8222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:56.185966 8236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:56.190735 8236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:56.201514 8236 catalog_manager.cc:1347] Generated new cluster ID: a5d80d76e93446ef9db2b0f0aa764ba8
I20260811 22:18:56.201730 8236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:56.223436 8236 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:18:56.225556 8236 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:56.252768 8236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 089f88d10e554230a5c5005d3617cacf: Generated new TSK 0
I20260811 22:18:56.253966 8236 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:18:56.336243 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:56.274037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48968 (local address 127.29.229.62:38701)
0811 22:18:56.274249 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:56.274256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:56.275140 (+ 884us) server_negotiation.cc:407] Connection header received
0811 22:18:56.275289 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:56.275294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:56.275370 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:56.275523 (+ 153us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:18:56.276102 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:56.277119 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:56.279286 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:56.279807 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:56.279920 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:56.280625 (+ 705us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:18:56.335590 (+ 54965us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 22:18:56.335948 (+ 358us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48968: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 22:18:56.336431 8216 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48968: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 22:18:56.336731 8262 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:18:56.273842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:38701 (local address 127.0.0.1:48968)
0811 22:18:56.274710 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0811 22:18:56.274739 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 22:18:56.274908 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:18:56.275571 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:18:56.275576 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:18:56.275590 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:18:56.275917 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:56.275930 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:56.277292 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:56.277298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:56.279099 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:18:56.279117 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:18:56.280001 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:18:56.280007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:18:56.280204 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:56.280258 (+ 54us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:18:56.335916 (+ 55658us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 22:18:56.336259 (+ 343us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.229.62:38701: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":534,"threads_started":1}
I20260811 22:18:56.337895 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 8084
I20260811 22:18:56.339335 8203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:18:56.445113 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 8084
2026-08-11T22:18:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (859 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (451 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (468 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:18:57 dist-test-slave-jq4t krb5kdc[8295](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:18:57 dist-test-slave-jq4t krb5kdc[8295](info): set up 2 sockets
Aug 11 22:18:57 dist-test-slave-jq4t krb5kdc[8295](info): commencing operation
krb5kdc: starting...
W20260811 22:18:58.726697 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:18:58 dist-test-slave-jq4t krb5kdc[8295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:18:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:18:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:18:59.035835 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:41421
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:42137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:41421
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:18:59.418610 8321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:18:59.419049 8321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:18:59.419256 8321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:18:59.429332 8321 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:18:59.429476 8321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:18:59.429581 8321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:18:59.429688 8321 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:18:59.429760 8321 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:18:59.442669 8321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:42137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:41421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:41421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:18:59.444578 8321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:18:59.446053 8321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:18:59.458367 8330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:59.458461 8333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:18:59.458531 8339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:18:59.459560 8321 server_base.cc:1034] running on GCE node
I20260811 22:18:59.460420 8321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:18:59.461956 8321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:18:59.463195 8321 hybrid_clock.cc:648] HybridClock initialized: now 1786486739463145 us; error 34 us; skew 500 ppm
I20260811 22:18:59.463615 8321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:18:59 dist-test-slave-jq4t krb5kdc[8295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486739, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:18:59.467005 8321 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:18:59.468739 8321 webserver.cc:460] Webserver started at http://127.29.229.62:40397/ using document root <none> and password file <none>
I20260811 22:18:59.469486 8321 fs_manager.cc:359] Metadata directory not provided
I20260811 22:18:59.469646 8321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:18:59.469993 8321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:18:59.473361 8321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "d9aa4f3546264e71bf9f58e5ad1fa723"
format_stamp: "Formatted at 2026-08-11 22:18:59 on dist-test-slave-jq4t"
server_key: "f470dc64f1c9371c4cb06b6ca66c472e"
server_key_iv: "14767acd98c6309624247ada2a63580d"
server_key_version: "encryptionkey@0"
I20260811 22:18:59.474222 8321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "d9aa4f3546264e71bf9f58e5ad1fa723"
format_stamp: "Formatted at 2026-08-11 22:18:59 on dist-test-slave-jq4t"
server_key: "f470dc64f1c9371c4cb06b6ca66c472e"
server_key_iv: "14767acd98c6309624247ada2a63580d"
server_key_version: "encryptionkey@0"
I20260811 22:18:59.479980 8321 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 22:18:59.484107 8359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:18:59.485622 8321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:18:59.485940 8321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "d9aa4f3546264e71bf9f58e5ad1fa723"
format_stamp: "Formatted at 2026-08-11 22:18:59 on dist-test-slave-jq4t"
server_key: "f470dc64f1c9371c4cb06b6ca66c472e"
server_key_iv: "14767acd98c6309624247ada2a63580d"
server_key_version: "encryptionkey@0"
I20260811 22:18:59.486172 8321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:18:59.526297 8321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:18:59.531749 8321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:18:59.532104 8321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:18:59.553692 8321 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:41421
I20260811 22:18:59.553745 8453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:41421 every 8 connection(s)
I20260811 22:18:59.555154 8321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:18:59.557665 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 8321
I20260811 22:18:59.557907 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:18:59.558804 30612 external_mini_cluster.cc:1428] Setting key de5af64edbe31d36669a41468c466d04
I20260811 22:18:59.559907 8458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:18:59.575376 8458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723: Bootstrap starting.
Aug 11 22:18:59 dist-test-slave-jq4t krb5kdc[8295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:18:59.578732 8458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723: Neither blocks nor log segments found. Creating new log.
I20260811 22:18:59.579851 8458 log.cc:824] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723: Log is configured to *not* fsync() on all Append() calls
I20260811 22:18:59.583655 8458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723: No bootstrap required, opened a new log
I20260811 22:18:59.584936 8466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:18:59.561338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36230 (local address 127.29.229.62:41421)
0811 22:18:59.562390 (+ 1052us) server_negotiation.cc:206] Beginning negotiation
0811 22:18:59.562401 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:18:59.562442 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:18:59.565049 (+ 2607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:18:59.565072 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:18:59.565485 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:18:59.565816 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:18:59.567694 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:59.569179 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:59.570965 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:18:59.571685 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:18:59.571835 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:18:59.576764 (+ 4929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:18:59.576772 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:18:59.576786 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:18:59.576830 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:18:59.580013 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:59.581009 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:59.581021 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:59.581032 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:59.581396 (+ 364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:18:59.581907 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:18:59.581911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:18:59.581913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:18:59.582481 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:18:59.582721 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 22:18:59.583196 (+ 475us) server_negotiation.cc:299] Negotiation successful
0811 22:18:59.583574 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":846,"thread_start_us":584,"threads_started":1}
I20260811 22:18:59.592449 8458 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9aa4f3546264e71bf9f58e5ad1fa723" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41421 } }
I20260811 22:18:59.592931 8458 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:18:59.593014 8458 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9aa4f3546264e71bf9f58e5ad1fa723, State: Initialized, Role: FOLLOWER
I20260811 22:18:59.593539 8458 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9aa4f3546264e71bf9f58e5ad1fa723" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41421 } }
I20260811 22:18:59.593779 8458 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:18:59.593952 8458 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:18:59.594139 8458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:18:59.596549 8458 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9aa4f3546264e71bf9f58e5ad1fa723" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41421 } }
I20260811 22:18:59.597114 8458 leader_election.cc:302] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9aa4f3546264e71bf9f58e5ad1fa723; no voters:
I20260811 22:18:59.597751 8458 leader_election.cc:288] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:18:59.597930 8468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:18:59.598640 8468 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723 [term 1 LEADER]: Becoming Leader. State: Replica: d9aa4f3546264e71bf9f58e5ad1fa723, State: Running, Role: LEADER
I20260811 22:18:59.599314 8468 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9aa4f3546264e71bf9f58e5ad1fa723" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41421 } }
I20260811 22:18:59.600004 8458 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:18:59.602552 8469 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9aa4f3546264e71bf9f58e5ad1fa723" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9aa4f3546264e71bf9f58e5ad1fa723" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41421 } } }
I20260811 22:18:59.603020 8469 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:59.602564 8472 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9aa4f3546264e71bf9f58e5ad1fa723. Latest consensus state: current_term: 1 leader_uuid: "d9aa4f3546264e71bf9f58e5ad1fa723" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9aa4f3546264e71bf9f58e5ad1fa723" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41421 } } }
I20260811 22:18:59.604769 8472 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723 [sys.catalog]: This master's current role is: LEADER
I20260811 22:18:59.605959 8482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:18:59.612754 8482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:18:59.622905 8482 catalog_manager.cc:1347] Generated new cluster ID: 5f99dd9b7deb402dad74e1821b8afc62
I20260811 22:18:59.623049 8482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:18:59.648339 8482 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:18:59.649776 8482 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:18:59.661515 8482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9aa4f3546264e71bf9f58e5ad1fa723: Generated new TSK 0
I20260811 22:18:59.662573 8482 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
I20260811 22:18:59.748819 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41421
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:42137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:00.144798 8502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:00.145192 8502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:00.145372 8502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:00.154425 8502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:00.154588 8502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:00.154841 8502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:19:00.166675 8502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:42137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:00.168233 8502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:00.169739 8502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:00.185436 8510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:00.185422 8514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:00.185540 8511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:00.186383 8502 server_base.cc:1034] running on GCE node
I20260811 22:19:00.187125 8502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:00.188180 8502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:00.189507 8502 hybrid_clock.cc:648] HybridClock initialized: now 1786486740189399 us; error 74 us; skew 500 ppm
I20260811 22:19:00.189893 8502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:00 dist-test-slave-jq4t krb5kdc[8295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486740, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:00.193480 8502 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:19:00.195129 8502 webserver.cc:460] Webserver started at http://127.29.229.1:42395/ using document root <none> and password file <none>
I20260811 22:19:00.195943 8502 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:00.196098 8502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:00.196422 8502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:00.199519 8502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "f6501c604ca24c4187d3bbd28ded0e7b"
format_stamp: "Formatted at 2026-08-11 22:19:00 on dist-test-slave-jq4t"
server_key: "20eed95be44ffbe89a93682654b75f28"
server_key_iv: "55812219929b147a66b1e69920b2423d"
server_key_version: "encryptionkey@0"
I20260811 22:19:00.200416 8502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "f6501c604ca24c4187d3bbd28ded0e7b"
format_stamp: "Formatted at 2026-08-11 22:19:00 on dist-test-slave-jq4t"
server_key: "20eed95be44ffbe89a93682654b75f28"
server_key_iv: "55812219929b147a66b1e69920b2423d"
server_key_version: "encryptionkey@0"
I20260811 22:19:00.206103 8502 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 22:19:00.210001 8529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:00.211464 8502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:19:00.211675 8502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "f6501c604ca24c4187d3bbd28ded0e7b"
format_stamp: "Formatted at 2026-08-11 22:19:00 on dist-test-slave-jq4t"
server_key: "20eed95be44ffbe89a93682654b75f28"
server_key_iv: "55812219929b147a66b1e69920b2423d"
server_key_version: "encryptionkey@0"
I20260811 22:19:00.211910 8502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:00.237655 8502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:00.242460 8502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:00.242789 8502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:00.243899 8502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:00.245389 8502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:00.245498 8502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:00.245651 8502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:00.245728 8502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:00.284126 8502 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:46731
I20260811 22:19:00.284219 8731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:46731 every 8 connection(s)
I20260811 22:19:00.285502 8502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:19:00.287855 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 8502
I20260811 22:19:00.288100 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:19:00.288952 30612 external_mini_cluster.cc:1428] Setting key 0ac4f371ce65d1c2b0b9420c7e9d7502
Aug 11 22:19:00 dist-test-slave-jq4t krb5kdc[8295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486740, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:19:00.307204 8734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.288696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:54401 (local address 127.29.229.62:41421)
0811 22:19:00.289217 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:00.289225 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:00.290452 (+ 1227us) server_negotiation.cc:407] Connection header received
0811 22:19:00.292808 (+ 2356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:00.292813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:00.292917 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:00.293039 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:00.294586 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.295327 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.296955 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.297381 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.297519 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.301272 (+ 3753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:00.301278 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:00.301281 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:00.301313 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:00.303975 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:00.304816 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:00.304821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:00.304824 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:00.304950 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:00.305411 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:00.305422 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:00.305426 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:00.305692 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:00.305864 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:00.306590 (+ 726us) server_negotiation.cc:299] Negotiation successful
0811 22:19:00.306814 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":251,"threads_started":1}
I20260811 22:19:00.308171 8735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.289308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41421 (local address 127.29.229.1:54401)
0811 22:19:00.290266 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:00.290321 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:00.292463 (+ 2142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:00.293289 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:00.293298 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:00.293823 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:00.294416 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:00.294434 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.295481 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.295486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:00.296796 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:00.296809 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.297632 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.297640 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:00.297868 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.298578 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:00.298600 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:00.301040 (+ 2440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:00.304129 (+ 3089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:00.304138 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:00.304155 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:00.304679 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:00.305070 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:00.305075 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:00.305078 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:00.305275 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:00.305835 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:00.305843 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:00.306187 (+ 344us) client_negotiation.cc:769] Sending connection context
0811 22:19:00.306465 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 22:19:00.307000 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":404,"threads_started":1}
I20260811 22:19:00.309845 8732 heartbeater.cc:347] Connected to a master server at 127.29.229.62:41421
I20260811 22:19:00.310155 8732 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:00.310932 8732 heartbeater.cc:511] Master 127.29.229.62:41421 requested a full tablet report, sending...
I20260811 22:19:00.313272 8395 ts_manager.cc:194] Registered new tserver with Master: f6501c604ca24c4187d3bbd28ded0e7b (127.29.229.1:46731)
I20260811 22:19:00.314774 8395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:54401
WARNING: no policy specified for kudu/127.29.229.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
I20260811 22:19:00.352836 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41421
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:42137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:00.713071 8740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:00.713378 8740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:00.713480 8740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:00.721385 8740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:00.721513 8740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:00.721632 8740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:19:00.731789 8740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:42137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:00.733139 8740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:00.734484 8740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:00.748165 8747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:00.748345 8751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:00.748364 8749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:00.749439 8740 server_base.cc:1034] running on GCE node
I20260811 22:19:00.750128 8740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:00.751145 8740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:00.752391 8740 hybrid_clock.cc:648] HybridClock initialized: now 1786486740752323 us; error 51 us; skew 500 ppm
I20260811 22:19:00.752787 8740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:00 dist-test-slave-jq4t krb5kdc[8295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486740, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:00.757118 8740 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:19:00.759333 8740 webserver.cc:460] Webserver started at http://127.29.229.2:41727/ using document root <none> and password file <none>
I20260811 22:19:00.760162 8740 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:00.760322 8740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:00.760718 8740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:00.763980 8740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "b2c7741dde5d4401b17ab6faba35142f"
format_stamp: "Formatted at 2026-08-11 22:19:00 on dist-test-slave-jq4t"
server_key: "16198186683ddb64cd6b472b367b8092"
server_key_iv: "9d1a0dc215e2ae25916f61def9bb5bd7"
server_key_version: "encryptionkey@0"
I20260811 22:19:00.764919 8740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "b2c7741dde5d4401b17ab6faba35142f"
format_stamp: "Formatted at 2026-08-11 22:19:00 on dist-test-slave-jq4t"
server_key: "16198186683ddb64cd6b472b367b8092"
server_key_iv: "9d1a0dc215e2ae25916f61def9bb5bd7"
server_key_version: "encryptionkey@0"
I20260811 22:19:00.770774 8740 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:19:00.774519 8761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:00.775956 8740 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:19:00.776170 8740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "b2c7741dde5d4401b17ab6faba35142f"
format_stamp: "Formatted at 2026-08-11 22:19:00 on dist-test-slave-jq4t"
server_key: "16198186683ddb64cd6b472b367b8092"
server_key_iv: "9d1a0dc215e2ae25916f61def9bb5bd7"
server_key_version: "encryptionkey@0"
I20260811 22:19:00.776403 8740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:00.793504 8740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:00.799005 8740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:00.799373 8740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:00.800884 8740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:00.802896 8740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:00.803020 8740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:00.803143 8740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:00.803215 8740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:00.855876 8740 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:39267
I20260811 22:19:00.855975 8958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:39267 every 8 connection(s)
I20260811 22:19:00.857692 8740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:19:00.864151 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 8740
I20260811 22:19:00.864362 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:19:00.864898 30612 external_mini_cluster.cc:1428] Setting key 3c33abac4217f14ee7416d011c51aab8
Aug 11 22:19:00 dist-test-slave-jq4t krb5kdc[8295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486740, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:19:00.882470 8966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.861334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:44371 (local address 127.29.229.62:41421)
0811 22:19:00.862036 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:00.862045 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:00.863547 (+ 1502us) server_negotiation.cc:407] Connection header received
0811 22:19:00.866789 (+ 3242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:00.866794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:00.866882 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:00.866996 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:00.868769 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.869519 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.871240 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.871703 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.871841 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.875701 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:00.875706 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:00.875709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:00.875745 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:00.878492 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:00.879305 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:00.879310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:00.879313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:00.879443 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:00.880027 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:00.880031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:00.880033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:00.880468 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:00.880658 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:00.881689 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0811 22:19:00.881958 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":341,"threads_started":1}
I20260811 22:19:00.883726 8967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:00.862173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41421 (local address 127.29.229.2:44371)
0811 22:19:00.863296 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:00.863368 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:00.866382 (+ 3014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:00.867354 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:00.867363 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:00.867920 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:00.868594 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:00.868615 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.869702 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.869706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:00.871088 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:00.871102 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:00.871964 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:00.871973 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:00.872254 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:00.873054 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:00.873079 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:00.875429 (+ 2350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:00.878658 (+ 3229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:00.878668 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:00.878688 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:00.879159 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:00.879561 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:00.879567 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:00.879570 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:00.879892 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:00.880655 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:00.880664 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:00.881197 (+ 533us) client_negotiation.cc:769] Sending connection context
0811 22:19:00.881556 (+ 359us) client_negotiation.cc:240] Negotiation successful
0811 22:19:00.882213 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":471,"threads_started":1}
I20260811 22:19:00.885612 8963 heartbeater.cc:347] Connected to a master server at 127.29.229.62:41421
I20260811 22:19:00.885972 8963 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:00.886868 8963 heartbeater.cc:511] Master 127.29.229.62:41421 requested a full tablet report, sending...
I20260811 22:19:00.888780 8395 ts_manager.cc:194] Registered new tserver with Master: b2c7741dde5d4401b17ab6faba35142f (127.29.229.2:39267)
I20260811 22:19:00.890131 8395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:44371
WARNING: no policy specified for kudu/127.29.229.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
I20260811 22:19:00.942481 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41421
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:42137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:01.303956 8970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:01.304319 8970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:01.304539 8970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:01.312125 8970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:01.312287 8970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:01.312536 8970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:19:01.318648 8732 heartbeater.cc:503] Master 127.29.229.62:41421 was elected leader, sending a full tablet report...
I20260811 22:19:01.323807 8970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:42137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:01.325217 8970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:01.326566 8970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:01.340516 8980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:01.340495 8981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:01.340432 8985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:01.341410 8970 server_base.cc:1034] running on GCE node
I20260811 22:19:01.342092 8970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:01.343256 8970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:01.344590 8970 hybrid_clock.cc:648] HybridClock initialized: now 1786486741344524 us; error 48 us; skew 500 ppm
I20260811 22:19:01.344959 8970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:01 dist-test-slave-jq4t krb5kdc[8295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486741, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:01.351678 8970 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:19:01.353432 8970 webserver.cc:460] Webserver started at http://127.29.229.3:41371/ using document root <none> and password file <none>
I20260811 22:19:01.354189 8970 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:01.354353 8970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:01.354691 8970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:01.357844 8970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "43523aab5cce402898f6c0f0dab65616"
format_stamp: "Formatted at 2026-08-11 22:19:01 on dist-test-slave-jq4t"
server_key: "5af83d6388f53a573c0d6c8e0d38148a"
server_key_iv: "1dd0f6ceec588cf6640a4eddc878929d"
server_key_version: "encryptionkey@0"
I20260811 22:19:01.358661 8970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "43523aab5cce402898f6c0f0dab65616"
format_stamp: "Formatted at 2026-08-11 22:19:01 on dist-test-slave-jq4t"
server_key: "5af83d6388f53a573c0d6c8e0d38148a"
server_key_iv: "1dd0f6ceec588cf6640a4eddc878929d"
server_key_version: "encryptionkey@0"
I20260811 22:19:01.364096 8970 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 22:19:01.367750 8996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:01.369251 8970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:19:01.369447 8970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "43523aab5cce402898f6c0f0dab65616"
format_stamp: "Formatted at 2026-08-11 22:19:01 on dist-test-slave-jq4t"
server_key: "5af83d6388f53a573c0d6c8e0d38148a"
server_key_iv: "1dd0f6ceec588cf6640a4eddc878929d"
server_key_version: "encryptionkey@0"
I20260811 22:19:01.369669 8970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:01.393384 8970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:01.398072 8970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:01.398391 8970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:01.400060 8970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:01.401916 8970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:01.402024 8970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:01.402187 8970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:01.402261 8970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:01.445803 8970 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:44325
I20260811 22:19:01.445930 9189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:44325 every 8 connection(s)
I20260811 22:19:01.447259 8970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:19:01.455909 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 8970
I20260811 22:19:01.456116 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:19:01.456760 30612 external_mini_cluster.cc:1428] Setting key 70d21749a2df107d162746a427123ea0
Aug 11 22:19:01 dist-test-slave-jq4t krb5kdc[8295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486741, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:19:01.474236 9195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.450395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:34153 (local address 127.29.229.62:41421)
0811 22:19:01.451171 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.451176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.452455 (+ 1279us) server_negotiation.cc:407] Connection header received
0811 22:19:01.454592 (+ 2137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.454598 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.454723 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.454863 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:01.457198 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.458236 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.460544 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.461049 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.461188 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.466352 (+ 5164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:01.466358 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:01.466362 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:01.466405 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:01.469537 (+ 3132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.470610 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.470618 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.470621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.470884 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.471631 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.471638 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.471642 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.472099 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:01.472273 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.473505 (+ 1232us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.473775 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":476,"threads_started":1}
I20260811 22:19:01.475281 9197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.451239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41421 (local address 127.29.229.3:34153)
0811 22:19:01.452241 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:01.452288 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:01.454276 (+ 1988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:01.455154 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:01.455164 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:01.456000 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:01.456960 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.456992 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.458473 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.458477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.460286 (+ 1809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:01.460303 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.461428 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.461435 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:01.461757 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.462727 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:01.462767 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:01.465994 (+ 3227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:01.469756 (+ 3762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:01.469765 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:01.469782 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:01.470333 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:01.471086 (+ 753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:01.471091 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:01.471095 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:01.471450 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:01.472316 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:01.472325 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:01.472979 (+ 654us) client_negotiation.cc:769] Sending connection context
0811 22:19:01.473369 (+ 390us) client_negotiation.cc:240] Negotiation successful
0811 22:19:01.474015 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":413,"threads_started":1}
I20260811 22:19:01.477133 9192 heartbeater.cc:347] Connected to a master server at 127.29.229.62:41421
I20260811 22:19:01.477504 9192 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:01.478312 9192 heartbeater.cc:511] Master 127.29.229.62:41421 requested a full tablet report, sending...
I20260811 22:19:01.480300 8395 ts_manager.cc:194] Registered new tserver with Master: 43523aab5cce402898f6c0f0dab65616 (127.29.229.3:44325)
I20260811 22:19:01.481529 8395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:34153
I20260811 22:19:01.487205 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/test-admin.keytab.
Aug 11 22:19:01 dist-test-slave-jq4t krb5kdc[8295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:01.526194 30612 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 22:19:01.532083 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:01.529006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33270 (local address 127.29.229.1:46731)
0811 22:19:01.529676 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.529684 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.529713 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:19:01.529836 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.529843 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.530065 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.530216 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:01.531633 (+ 1417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33270: BlockingRecv error: recv got EOF from 127.0.0.1:33270 (error 108)
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":320,"threads_started":1}
W20260811 22:19:01.532568 30612 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/krb5cc' of type FILE
Aug 11 22:19:01 dist-test-slave-jq4t krb5kdc[8295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:01.534500 30612 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:19:01 dist-test-slave-jq4t krb5kdc[8295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:01.536099 30612 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:19:01 dist-test-slave-jq4t krb5kdc[8295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:19:01.553882 9207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.538010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33274 (local address 127.29.229.1:46731)
0811 22:19:01.538222 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.538228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.538603 (+ 375us) server_negotiation.cc:407] Connection header received
0811 22:19:01.538735 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.538740 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.538862 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.539021 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:01.540163 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.541305 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.542687 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.543332 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.543447 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.547588 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:01.547601 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:01.547617 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:01.547659 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:01.550594 (+ 2935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.551306 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.551315 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.551318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.551461 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.551886 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.551891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.551893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.552447 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:01.552788 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.553200 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.553574 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 22:19:01.562533 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:01.560198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33282 (local address 127.29.229.1:46731)
0811 22:19:01.560406 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.560412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.560776 (+ 364us) server_negotiation.cc:407] Connection header received
0811 22:19:01.560956 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.560960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.561051 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.561169 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:01.562227 (+ 1058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33282: BlockingRecv error: recv got EOF from 127.0.0.1:33282 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260811 22:19:01.563038 30612 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/krb5cc)
Aug 11 22:19:01 dist-test-slave-jq4t krb5kdc[8295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:01.564754 30612 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:19:01 dist-test-slave-jq4t krb5kdc[8295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:01.566056 30612 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:19:01 dist-test-slave-jq4t krb5kdc[8295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:19:01.582480 9207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.568208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33290 (local address 127.29.229.1:46731)
0811 22:19:01.568463 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.568506 (+ 43us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.568797 (+ 291us) server_negotiation.cc:407] Connection header received
0811 22:19:01.569033 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.569037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.569109 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.569225 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:01.570206 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.570928 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.572132 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.572659 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.572813 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.576801 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:01.576806 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:01.576810 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:01.576847 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:01.579592 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.580449 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.580459 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.580461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.580644 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.581044 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.581049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.581052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.581386 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:01.581656 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.581967 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.582186 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
W20260811 22:19:01.588392 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:01.586025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33300 (local address 127.29.229.1:46731)
0811 22:19:01.586302 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.586312 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.586833 (+ 521us) server_negotiation.cc:407] Connection header received
0811 22:19:01.586959 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.586964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.587041 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.587233 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:01.588107 (+ 874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33300: BlockingRecv error: recv got EOF from 127.0.0.1:33300 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
W20260811 22:19:01.588399 30612 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486685119861-30612-0/krb5kdc/krb5cc' of type FILE
Aug 11 22:19:01 dist-test-slave-jq4t krb5kdc[8295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:01.590196 30612 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:19:01 dist-test-slave-jq4t krb5kdc[8295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:01.592410 30612 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:19:01 dist-test-slave-jq4t krb5kdc[8295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:19:01.610451 9207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.594194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33304 (local address 127.29.229.1:46731)
0811 22:19:01.594374 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.594378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.594817 (+ 439us) server_negotiation.cc:407] Connection header received
0811 22:19:01.595015 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.595026 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.595090 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.595195 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:01.596205 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.597292 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.598815 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.599279 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.599444 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.603402 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:01.603408 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:01.603412 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:01.603469 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:01.606479 (+ 3010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.607573 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.607578 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.607580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.607749 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.608358 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.608362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.608364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.608873 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:01.609138 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.609852 (+ 714us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.610096 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:19:01.612884 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 8502
I20260811 22:19:01.626464 9207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.613816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33316 (local address 127.29.229.1:46731)
0811 22:19:01.613991 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.613996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.614457 (+ 461us) server_negotiation.cc:407] Connection header received
0811 22:19:01.614667 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.614672 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.614745 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.614853 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:01.616008 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.616923 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.618127 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.618576 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.618718 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.621116 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:01.621120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:01.621124 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:01.621158 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:01.623758 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.624421 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.624428 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.624436 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.624627 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.625016 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.625021 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.625025 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.625306 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:01.625618 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.625902 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.626137 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:19:01.627125 8726 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:01.716598 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 8502
I20260811 22:19:01.745314 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 8740
Aug 11 22:19:01 dist-test-slave-jq4t krb5kdc[8295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.2@KRBTEST.COM
I20260811 22:19:01.761886 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.746343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38552 (local address 127.29.229.2:39267)
0811 22:19:01.746939 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.746946 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.746973 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:19:01.747036 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.747042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.747258 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.747405 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:01.748449 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.749566 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.750768 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.751367 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.751478 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.755688 (+ 4210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:01.755702 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:01.755718 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:01.755755 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:01.758914 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.759666 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.759674 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.759676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.759811 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.760253 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.760257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.760259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.760749 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:01.760997 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.761362 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.761579 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":329,"threads_started":1}
I20260811 22:19:01.763217 8950 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:01.849009 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 8740
I20260811 22:19:01.875674 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 8970
Aug 11 22:19:01 dist-test-slave-jq4t krb5kdc[8295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.3@KRBTEST.COM
I20260811 22:19:01.895300 9228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:01.877195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39008 (local address 127.29.229.3:44325)
0811 22:19:01.877770 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:01.877778 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:01.877808 (+ 30us) server_negotiation.cc:407] Connection header received
0811 22:19:01.877872 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:01.877878 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:01.878112 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:01.878304 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:01.880010 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.881662 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.883265 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:01.883935 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:01.884102 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:01.889055 (+ 4953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:01.889071 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:01.889087 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:01.889137 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:01.892245 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.893100 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.893108 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.893111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.893245 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:01.893654 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:01.893659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:01.893662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:01.894053 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:01.894369 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:01.894681 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 22:19:01.894973 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":265,"threads_started":1}
I20260811 22:19:01.896741 9184 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:01.983539 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 8970
I20260811 22:19:02.012343 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 8321
I20260811 22:19:02.013953 8448 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:02.122927 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 8321
2026-08-11T22:19:02Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4765 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:19:02 dist-test-slave-jq4t krb5kdc[9237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:19:02 dist-test-slave-jq4t krb5kdc[9237](info): set up 2 sockets
Aug 11 22:19:02 dist-test-slave-jq4t krb5kdc[9237](info): commencing operation
krb5kdc: starting...
W20260811 22:19:03.520327 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.341s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:19:03 dist-test-slave-jq4t krb5kdc[9237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:19:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:19:03Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.62.keytab.
Entry for principal oryx/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.62.keytab.
I20260811 22:19:03.833022 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:43683
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.62.keytab
--principal=oryx/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:43683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:04.203015 9255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:04.203331 9255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:04.203510 9255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:04.212379 9255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:19:04.212589 9255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:04.212704 9255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:04.212811 9255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:19:04.212898 9255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:19:04.223356 9255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:43683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.62.keytab
--principal=oryx/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:43683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:04.224944 9255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:04.226369 9255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:04.240204 9270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:04.240243 9272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:04.240274 9264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:04.241086 9255 server_base.cc:1034] running on GCE node
I20260811 22:19:04.241869 9255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:04.243360 9255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:04.244647 9255 hybrid_clock.cc:648] HybridClock initialized: now 1786486744244593 us; error 43 us; skew 500 ppm
I20260811 22:19:04.245002 9255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:04 dist-test-slave-jq4t krb5kdc[9237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486744, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:04.248237 9255 init.cc:377] Logged in from keytab as oryx/127.29.229.62@KRBTEST.COM (short username oryx)
I20260811 22:19:04.250155 9255 webserver.cc:460] Webserver started at http://127.29.229.62:36829/ using document root <none> and password file <none>
I20260811 22:19:04.250903 9255 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:04.251048 9255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:04.251405 9255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:04.254755 9255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "fc323e8e063549859af13dc2f68c4f2f"
format_stamp: "Formatted at 2026-08-11 22:19:04 on dist-test-slave-jq4t"
server_key: "7e6c9100c4e1844644c28e61da342cf5"
server_key_iv: "975c31eb92d1934c8217ed5f689a0271"
server_key_version: "encryptionkey@0"
I20260811 22:19:04.255600 9255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "fc323e8e063549859af13dc2f68c4f2f"
format_stamp: "Formatted at 2026-08-11 22:19:04 on dist-test-slave-jq4t"
server_key: "7e6c9100c4e1844644c28e61da342cf5"
server_key_iv: "975c31eb92d1934c8217ed5f689a0271"
server_key_version: "encryptionkey@0"
I20260811 22:19:04.261902 9255 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 22:19:04.266950 9281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:04.268548 9255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 22:19:04.268766 9255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "fc323e8e063549859af13dc2f68c4f2f"
format_stamp: "Formatted at 2026-08-11 22:19:04 on dist-test-slave-jq4t"
server_key: "7e6c9100c4e1844644c28e61da342cf5"
server_key_iv: "975c31eb92d1934c8217ed5f689a0271"
server_key_version: "encryptionkey@0"
I20260811 22:19:04.268993 9255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:04.296396 9255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:04.301599 9255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:04.301961 9255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:04.323666 9255 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:43683
I20260811 22:19:04.323705 9367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:43683 every 8 connection(s)
I20260811 22:19:04.325693 9255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:19:04.330427 9371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:04.335878 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 9255
I20260811 22:19:04.336103 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:19:04.336908 30612 external_mini_cluster.cc:1428] Setting key 5446bb2aeecbae6c6ee8a44bf01e06df
I20260811 22:19:04.343060 9371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f: Bootstrap starting.
I20260811 22:19:04.347057 9371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:04.348342 9371 log.cc:824] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:04.352624 9371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f: No bootstrap required, opened a new log
Aug 11 22:19:04 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM
I20260811 22:19:04.360771 9371 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } }
I20260811 22:19:04.361195 9371 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:04.361328 9371 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc323e8e063549859af13dc2f68c4f2f, State: Initialized, Role: FOLLOWER
I20260811 22:19:04.361910 9371 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } }
I20260811 22:19:04.362179 9371 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:19:04.362339 9371 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:19:04.362525 9371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:04.362743 9376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:04.339156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44596 (local address 127.29.229.62:43683)
0811 22:19:04.340081 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:04.340096 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:04.340140 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:19:04.341987 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:04.342019 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:04.342422 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:04.342766 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:04.344823 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:04.346189 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:04.348019 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:04.348930 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:04.349444 (+ 514us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:04.354399 (+ 4955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:04.354410 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:04.354436 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:04.354499 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:04.358129 (+ 3630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:04.358917 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:04.358928 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:04.358935 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:04.359142 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:04.359544 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:04.359551 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:04.359555 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:04.360138 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:04.360456 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:04.360936 (+ 480us) server_negotiation.cc:299] Negotiation successful
0811 22:19:04.361459 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":429,"threads_started":1}
I20260811 22:19:04.365175 9371 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } }
I20260811 22:19:04.365693 9371 leader_election.cc:302] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc323e8e063549859af13dc2f68c4f2f; no voters:
I20260811 22:19:04.366358 9371 leader_election.cc:288] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:19:04.366534 9378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:19:04.367589 9378 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [term 1 LEADER]: Becoming Leader. State: Replica: fc323e8e063549859af13dc2f68c4f2f, State: Running, Role: LEADER
I20260811 22:19:04.368299 9378 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } }
I20260811 22:19:04.368760 9371 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:19:04.372887 9383 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc323e8e063549859af13dc2f68c4f2f. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } }
I20260811 22:19:04.373276 9383 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:04.373109 9380 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } }
I20260811 22:19:04.373564 9380 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:04.378089 9391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:19:04.383484 9391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:19:04.394301 9391 catalog_manager.cc:1347] Generated new cluster ID: fdc9af6459954353a14b059d59b80b61
I20260811 22:19:04.394448 9391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:19:04.409140 9391 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:19:04.410547 9391 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:19:04.418612 9391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f: Generated new TSK 0
I20260811 22:19:04.419842 9391 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.1.keytab.
Entry for principal oryx/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.1.keytab.
I20260811 22:19:04.488511 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43683
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.1.keytab
--principal=oryx/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:04.883968 9416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:04.884284 9416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:04.884430 9416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:04.892145 9416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:04.892292 9416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:04.892526 9416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:19:04.902958 9416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.1.keytab
--principal=oryx/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:04.904436 9416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:04.906018 9416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:04.920933 9428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:04.921046 9423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:04.921010 9425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:04.922271 9416 server_base.cc:1034] running on GCE node
I20260811 22:19:04.922919 9416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:04.923883 9416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:04.925163 9416 hybrid_clock.cc:648] HybridClock initialized: now 1786486744925111 us; error 67 us; skew 500 ppm
I20260811 22:19:04.925526 9416 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:04 dist-test-slave-jq4t krb5kdc[9237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486744, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:04.928907 9416 init.cc:377] Logged in from keytab as oryx/127.29.229.1@KRBTEST.COM (short username oryx)
I20260811 22:19:04.930573 9416 webserver.cc:460] Webserver started at http://127.29.229.1:34097/ using document root <none> and password file <none>
I20260811 22:19:04.931327 9416 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:04.931486 9416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:04.931815 9416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:04.935128 9416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "be34d3c2b4c5470caf70aac065cc5d31"
format_stamp: "Formatted at 2026-08-11 22:19:04 on dist-test-slave-jq4t"
server_key: "b1a0a9414656780a5e6297e3f3241449"
server_key_iv: "acbdfe9cd38e695274cb06d2f8ee679f"
server_key_version: "encryptionkey@0"
I20260811 22:19:04.936021 9416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "be34d3c2b4c5470caf70aac065cc5d31"
format_stamp: "Formatted at 2026-08-11 22:19:04 on dist-test-slave-jq4t"
server_key: "b1a0a9414656780a5e6297e3f3241449"
server_key_iv: "acbdfe9cd38e695274cb06d2f8ee679f"
server_key_version: "encryptionkey@0"
I20260811 22:19:04.941653 9416 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 22:19:04.945537 9441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:04.947137 9416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:19:04.947342 9416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "be34d3c2b4c5470caf70aac065cc5d31"
format_stamp: "Formatted at 2026-08-11 22:19:04 on dist-test-slave-jq4t"
server_key: "b1a0a9414656780a5e6297e3f3241449"
server_key_iv: "acbdfe9cd38e695274cb06d2f8ee679f"
server_key_version: "encryptionkey@0"
I20260811 22:19:04.947592 9416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:04.973042 9416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:04.977772 9416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:04.978096 9416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:04.979362 9416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:04.980896 9416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:04.980988 9416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:04.981078 9416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:04.981158 9416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:05.020712 9416 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:45275
I20260811 22:19:05.020771 9636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:45275 every 8 connection(s)
I20260811 22:19:05.022138 9416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:19:05.026458 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 9416
I20260811 22:19:05.026708 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:19:05.027604 30612 external_mini_cluster.cc:1428] Setting key 9b8a836b6c7c52207448bdc9d90e3e63
Aug 11 22:19:05 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486744, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM
I20260811 22:19:05.049396 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:05.025214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:59435 (local address 127.29.229.62:43683)
0811 22:19:05.025727 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:05.025734 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:05.026949 (+ 1215us) server_negotiation.cc:407] Connection header received
0811 22:19:05.030103 (+ 3154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:05.030112 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:05.030251 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:05.030377 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:05.032812 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:05.034091 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:05.036237 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:05.037021 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:05.037207 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:05.041883 (+ 4676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:05.041894 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:05.041902 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:05.041956 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:05.045251 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:05.046384 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:05.046388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:05.046391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:05.046531 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:05.047066 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:05.047070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:05.047072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:05.047390 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:05.047525 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:05.048625 (+ 1100us) server_negotiation.cc:299] Negotiation successful
0811 22:19:05.048828 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":242,"threads_started":1}
I20260811 22:19:05.051381 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:05.025721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.1:59435)
0811 22:19:05.026654 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:05.026720 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:05.029488 (+ 2768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:05.030833 (+ 1345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:05.030845 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:05.031577 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:05.032467 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:05.032574 (+ 107us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:05.034521 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:05.034527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:05.035967 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:05.035988 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:05.037415 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:05.037422 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:05.037765 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:05.038865 (+ 1100us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:05.038897 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:05.041579 (+ 2682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:05.045456 (+ 3877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:05.045470 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:05.045492 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:05.046203 (+ 711us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:05.046652 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:05.046660 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:05.046667 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:05.046929 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:05.047547 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:05.047559 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:05.048075 (+ 516us) client_negotiation.cc:769] Sending connection context
0811 22:19:05.048420 (+ 345us) client_negotiation.cc:240] Negotiation successful
0811 22:19:05.049148 (+ 728us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":352,"threads_started":1}
I20260811 22:19:05.053589 9637 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43683
I20260811 22:19:05.054481 9637 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:05.056180 9637 heartbeater.cc:511] Master 127.29.229.62:43683 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.29.229.2@KRBTEST.COM; defaulting to no policy
I20260811 22:19:05.065172 9310 ts_manager.cc:194] Registered new tserver with Master: be34d3c2b4c5470caf70aac065cc5d31 (127.29.229.1:45275)
I20260811 22:19:05.067402 9310 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:59435
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.229.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.2.keytab.
Entry for principal oryx/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.2.keytab.
I20260811 22:19:05.107467 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43683
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.2.keytab
--principal=oryx/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:05.495991 9644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:05.496296 9644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:05.496455 9644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:05.504032 9644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:05.504228 9644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:05.504421 9644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:19:05.514830 9644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.2.keytab
--principal=oryx/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:05.516317 9644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:05.517704 9644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:05.531354 9651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:05.531368 9654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:05.531394 9656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:05.532706 9644 server_base.cc:1034] running on GCE node
I20260811 22:19:05.533420 9644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:05.534386 9644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:05.535653 9644 hybrid_clock.cc:648] HybridClock initialized: now 1786486745535593 us; error 44 us; skew 500 ppm
I20260811 22:19:05.535984 9644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:05 dist-test-slave-jq4t krb5kdc[9237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486745, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:05.539649 9644 init.cc:377] Logged in from keytab as oryx/127.29.229.2@KRBTEST.COM (short username oryx)
I20260811 22:19:05.541518 9644 webserver.cc:460] Webserver started at http://127.29.229.2:33195/ using document root <none> and password file <none>
I20260811 22:19:05.542276 9644 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:05.542425 9644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:05.542769 9644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:05.545949 9644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "655348f7beb645b98a09af862a80c1ee"
format_stamp: "Formatted at 2026-08-11 22:19:05 on dist-test-slave-jq4t"
server_key: "a84e269e4772a74fb592df63e34d5771"
server_key_iv: "e14a4d12b5f3d637d82c7594d16cfdd5"
server_key_version: "encryptionkey@0"
I20260811 22:19:05.546799 9644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "655348f7beb645b98a09af862a80c1ee"
format_stamp: "Formatted at 2026-08-11 22:19:05 on dist-test-slave-jq4t"
server_key: "a84e269e4772a74fb592df63e34d5771"
server_key_iv: "e14a4d12b5f3d637d82c7594d16cfdd5"
server_key_version: "encryptionkey@0"
I20260811 22:19:05.552867 9644 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 22:19:05.557394 9682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:05.559585 9644 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:19:05.559803 9644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "655348f7beb645b98a09af862a80c1ee"
format_stamp: "Formatted at 2026-08-11 22:19:05 on dist-test-slave-jq4t"
server_key: "a84e269e4772a74fb592df63e34d5771"
server_key_iv: "e14a4d12b5f3d637d82c7594d16cfdd5"
server_key_version: "encryptionkey@0"
I20260811 22:19:05.560007 9644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:05.571233 9644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:05.576695 9644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:05.577051 9644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:05.578603 9644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:05.580222 9644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:05.580300 9644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:05.580441 9644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:05.580547 9644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 22:19:05.619118 9644 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:41123
I20260811 22:19:05.619223 9860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:41123 every 8 connection(s)
I20260811 22:19:05.620801 9644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:19:05.627170 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 9644
I20260811 22:19:05.627394 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:19:05.628106 30612 external_mini_cluster.cc:1428] Setting key 82640cb46d588d659fb8f549c9677d5b
Aug 11 22:19:05 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486745, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM
I20260811 22:19:05.647521 9865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:05.623804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:42977 (local address 127.29.229.62:43683)
0811 22:19:05.624315 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:05.624320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:05.625403 (+ 1083us) server_negotiation.cc:407] Connection header received
0811 22:19:05.627320 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:05.627326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:05.627426 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:05.627568 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:05.630325 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:05.631218 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:05.633990 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:05.634488 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:05.634595 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:05.640015 (+ 5420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:05.640022 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:05.640025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:05.640068 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:05.643966 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:05.645000 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:05.645006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:05.645009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:05.645156 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:05.645624 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:05.645637 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:05.645640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:05.645940 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:05.646106 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:05.646874 (+ 768us) server_negotiation.cc:299] Negotiation successful
0811 22:19:05.647126 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":289,"threads_started":1}
I20260811 22:19:05.648615 9866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:05.624296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.2:42977)
0811 22:19:05.625222 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:05.625261 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:05.627042 (+ 1781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:05.627958 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:05.627971 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:05.628781 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:05.630020 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:05.630059 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:05.631413 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:05.631421 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:05.633797 (+ 2376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:05.633827 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:05.634761 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:05.634771 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:05.635130 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:05.636093 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:05.636130 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:05.639603 (+ 3473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:05.644184 (+ 4581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:05.644196 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:05.644223 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:05.644847 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:05.645280 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:05.645285 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:05.645288 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:05.645495 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:05.646077 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:05.646085 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:05.646463 (+ 378us) client_negotiation.cc:769] Sending connection context
0811 22:19:05.646753 (+ 290us) client_negotiation.cc:240] Negotiation successful
0811 22:19:05.647344 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":445,"threads_started":1}
I20260811 22:19:05.650243 9861 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43683
I20260811 22:19:05.650627 9861 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:05.651664 9861 heartbeater.cc:511] Master 127.29.229.62:43683 requested a full tablet report, sending...
I20260811 22:19:05.653492 9310 ts_manager.cc:194] Registered new tserver with Master: 655348f7beb645b98a09af862a80c1ee (127.29.229.2:41123)
I20260811 22:19:05.654809 9310 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:42977
WARNING: no policy specified for oryx/127.29.229.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.3.keytab.
Entry for principal oryx/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.3.keytab.
I20260811 22:19:05.696944 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43683
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.3.keytab
--principal=oryx/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:06.066061 9869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:06.066542 9869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:06.066717 9869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 22:19:06.072358 9637 heartbeater.cc:503] Master 127.29.229.62:43683 was elected leader, sending a full tablet report...
W20260811 22:19:06.075627 9869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:06.075757 9869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:06.076195 9869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:19:06.088897 9869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.3.keytab
--principal=oryx/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:06.090270 9869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:06.091667 9869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:06.105580 9879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:06.106124 9877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:06.106261 9881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:06.106652 9869 server_base.cc:1034] running on GCE node
I20260811 22:19:06.107332 9869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:06.108366 9869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:06.109622 9869 hybrid_clock.cc:648] HybridClock initialized: now 1786486746109556 us; error 57 us; skew 500 ppm
I20260811 22:19:06.109985 9869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:06 dist-test-slave-jq4t krb5kdc[9237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486746, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:06.113468 9869 init.cc:377] Logged in from keytab as oryx/127.29.229.3@KRBTEST.COM (short username oryx)
I20260811 22:19:06.115053 9869 webserver.cc:460] Webserver started at http://127.29.229.3:38879/ using document root <none> and password file <none>
I20260811 22:19:06.115803 9869 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:06.115975 9869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:06.116318 9869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:06.119524 9869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "43ca70cae5964345831bcafd0d7178bf"
format_stamp: "Formatted at 2026-08-11 22:19:06 on dist-test-slave-jq4t"
server_key: "94a0af77ba2d9c81b6379716056c4660"
server_key_iv: "a3627404d642a5f3156a31921817cd97"
server_key_version: "encryptionkey@0"
I20260811 22:19:06.120407 9869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "43ca70cae5964345831bcafd0d7178bf"
format_stamp: "Formatted at 2026-08-11 22:19:06 on dist-test-slave-jq4t"
server_key: "94a0af77ba2d9c81b6379716056c4660"
server_key_iv: "a3627404d642a5f3156a31921817cd97"
server_key_version: "encryptionkey@0"
I20260811 22:19:06.126156 9869 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 22:19:06.130621 9891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:06.132231 9869 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:19:06.132434 9869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "43ca70cae5964345831bcafd0d7178bf"
format_stamp: "Formatted at 2026-08-11 22:19:06 on dist-test-slave-jq4t"
server_key: "94a0af77ba2d9c81b6379716056c4660"
server_key_iv: "a3627404d642a5f3156a31921817cd97"
server_key_version: "encryptionkey@0"
I20260811 22:19:06.132720 9869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:06.145139 9869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:06.149833 9869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:06.150167 9869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:06.151224 9869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:06.152899 9869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:06.153017 9869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:06.153237 9869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:06.153373 9869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:06.192956 9869 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:46419
I20260811 22:19:06.192992 10084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:46419 every 8 connection(s)
I20260811 22:19:06.194303 9869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:19:06.199231 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 9869
I20260811 22:19:06.199478 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:19:06.200420 30612 external_mini_cluster.cc:1428] Setting key be8a855d9007b6ab9c1dbd3c2f466c4a
Aug 11 22:19:06 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486746, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM
I20260811 22:19:06.220722 10089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:06.197179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:51489 (local address 127.29.229.62:43683)
0811 22:19:06.197696 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:06.197701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:06.199199 (+ 1498us) server_negotiation.cc:407] Connection header received
0811 22:19:06.201682 (+ 2483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:06.201686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:06.201758 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:06.201907 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:06.204003 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.205039 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.207705 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.208391 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.208719 (+ 328us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:06.213004 (+ 4285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:06.213010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:06.213014 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:06.213067 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:06.217021 (+ 3954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:06.218177 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:06.218182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:06.218186 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:06.218340 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:06.218798 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:06.218811 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:06.218814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:06.219137 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:06.219263 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:06.220105 (+ 842us) server_negotiation.cc:299] Negotiation successful
0811 22:19:06.220306 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":269,"threads_started":1}
I20260811 22:19:06.221771 10090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:06.197838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.3:51489)
0811 22:19:06.198887 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:06.198953 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:06.201336 (+ 2383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:06.202216 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:06.202225 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:06.202841 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:06.203734 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:06.203762 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.205307 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.205314 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:06.207501 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:06.207531 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.208902 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.208918 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:06.209172 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:06.210000 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:06.210025 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:06.212710 (+ 2685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:06.217224 (+ 4514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:06.217237 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:06.217266 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:06.218029 (+ 763us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:06.218467 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:06.218471 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:06.218473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:06.218668 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:06.219307 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:06.219314 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:06.219707 (+ 393us) client_negotiation.cc:769] Sending connection context
0811 22:19:06.219993 (+ 286us) client_negotiation.cc:240] Negotiation successful
0811 22:19:06.220449 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":385,"threads_started":1}
I20260811 22:19:06.223320 10086 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43683
I20260811 22:19:06.223714 10086 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:06.224655 10086 heartbeater.cc:511] Master 127.29.229.62:43683 requested a full tablet report, sending...
I20260811 22:19:06.226387 9310 ts_manager.cc:194] Registered new tserver with Master: 43ca70cae5964345831bcafd0d7178bf (127.29.229.3:46419)
I20260811 22:19:06.227269 9310 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:51489
I20260811 22:19:06.231117 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.229.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.229.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.229.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.61.keytab.
Entry for principal oryx/127.29.229.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.61.keytab.
I20260811 22:19:06.306264 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.61:45437
--webserver_interface=127.29.229.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.61.keytab
--principal=oryx/127.29.229.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:43683,127.29.229.61:45437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:19:06.658517 9861 heartbeater.cc:503] Master 127.29.229.62:43683 was elected leader, sending a full tablet report...
W20260811 22:19:06.687604 10093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:06.688020 10093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:06.688199 10093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:06.697422 10093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:19:06.697638 10093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:06.697765 10093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:06.697882 10093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:19:06.697973 10093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:19:06.711027 10093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:43683,127.29.229.61:45437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.61.keytab
--principal=oryx/127.29.229.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.61:45437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.229.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
W20260811 22:19:06.711647 10093 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.29.229.62:43683,127.29.229.61:45437'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 22:19:06.713267 10093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:06.714998 10093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:06.730469 10098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:06.730643 10101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:06.731164 10099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:06.732468 10093 server_base.cc:1034] running on GCE node
I20260811 22:19:06.733861 10093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:06.735571 10093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:06.736900 10093 hybrid_clock.cc:648] HybridClock initialized: now 1786486746736807 us; error 61 us; skew 500 ppm
I20260811 22:19:06.737370 10093 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:06 dist-test-slave-jq4t krb5kdc[9237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486746, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:06.746842 10093 init.cc:377] Logged in from keytab as oryx/127.29.229.61@KRBTEST.COM (short username oryx)
I20260811 22:19:06.749375 10093 webserver.cc:460] Webserver started at http://127.29.229.61:43003/ using document root <none> and password file <none>
I20260811 22:19:06.750566 10093 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:06.750697 10093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:06.751066 10093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:06.754581 10093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/data/instance:
uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79"
format_stamp: "Formatted at 2026-08-11 22:19:06 on dist-test-slave-jq4t"
server_key: "b9698c452e0aef98c67ddcec0da5553b"
server_key_iv: "88693342d6e7fbabe4bed6bf9d8fad1b"
server_key_version: "encryptionkey@0"
I20260811 22:19:06.755374 10093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/wal/instance:
uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79"
format_stamp: "Formatted at 2026-08-11 22:19:06 on dist-test-slave-jq4t"
server_key: "b9698c452e0aef98c67ddcec0da5553b"
server_key_iv: "88693342d6e7fbabe4bed6bf9d8fad1b"
server_key_version: "encryptionkey@0"
I20260811 22:19:06.761166 10093 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 22:19:06.765268 10111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:06.766856 10093 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:19:06.767084 10093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/wal
uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79"
format_stamp: "Formatted at 2026-08-11 22:19:06 on dist-test-slave-jq4t"
server_key: "b9698c452e0aef98c67ddcec0da5553b"
server_key_iv: "88693342d6e7fbabe4bed6bf9d8fad1b"
server_key_version: "encryptionkey@0"
I20260811 22:19:06.767339 10093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:06.782593 10093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:06.787341 10093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:06.787667 10093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:06.809682 10093 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.61:45437
I20260811 22:19:06.809727 10196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.61:45437 every 8 connection(s)
I20260811 22:19:06.811249 10093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/data/info.pb
I20260811 22:19:06.815944 10197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:06.820281 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 10093
I20260811 22:19:06.824312 10197 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:19:06 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.229.61@KRBTEST.COM
Aug 11 22:19:06 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486746, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.61@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM
I20260811 22:19:06.845768 10206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:06.828460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57670 (local address 127.29.229.62:43683)
0811 22:19:06.829801 (+ 1341us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:06.829806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:06.829822 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:19:06.829876 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:06.829879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:06.829939 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:06.830068 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:06.831328 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.832056 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.833714 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.834498 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.834656 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:06.839141 (+ 4485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:06.839146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:06.839150 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:06.839183 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:06.842299 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:06.843319 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:06.843328 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:06.843332 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:06.843491 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:06.843935 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:06.843939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:06.843941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:06.844416 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:06.844611 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:06.845333 (+ 722us) server_negotiation.cc:299] Negotiation successful
0811 22:19:06.845517 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1243,"thread_start_us":1095,"threads_started":1}
I20260811 22:19:06.846985 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:06.822918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60480 (local address 127.29.229.61:45437)
0811 22:19:06.823828 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:06.823841 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:06.823886 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:19:06.825717 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:06.825741 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:06.826186 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:06.826629 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:06.828244 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.830437 (+ 2193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.831776 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.832719 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.833010 (+ 291us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:06.838288 (+ 5278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:06.838310 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:06.838327 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:06.838364 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:06.841677 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:06.842591 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:06.842603 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:06.842614 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:06.842810 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:06.843390 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:06.843393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:06.843395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:06.844115 (+ 720us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:06.844354 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:06.844953 (+ 599us) server_negotiation.cc:299] Negotiation successful
0811 22:19:06.845507 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":676,"thread_start_us":442,"threads_started":1}
I20260811 22:19:06.847007 10205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:06.828391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.0.0.1:57670)
0811 22:19:06.829208 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:06.829252 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:06.829406 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:06.830144 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:06.830152 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:06.830890 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:06.831156 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:06.831165 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.832211 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.832224 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:06.833491 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:06.833511 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.834761 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.834766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:06.834923 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:06.835577 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:06.835605 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:06.838921 (+ 3316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:06.842440 (+ 3519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:06.842450 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:06.842460 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:06.843088 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:06.843605 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:06.843610 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:06.843613 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:06.843770 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:06.844578 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:06.844585 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:06.844834 (+ 249us) client_negotiation.cc:769] Sending connection context
0811 22:19:06.845101 (+ 267us) client_negotiation.cc:240] Negotiation successful
0811 22:19:06.845629 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"mutex_wait_us":149,"spinlock_wait_cycles":21888,"thread_start_us":384,"threads_started":1}
I20260811 22:19:06.849653 10197 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:19:06 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486746, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.61@KRBTEST.COM for oryx/127.29.229.61@KRBTEST.COM
I20260811 22:19:06.863587 10205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:06.850352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45437 (local address 127.0.0.1:60486)
0811 22:19:06.850530 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:06.850550 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:06.850642 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:06.851069 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:06.851073 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:06.851381 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:06.851511 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:06.851519 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.852704 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.852708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:06.853649 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:06.853660 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.854417 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.854421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:06.854584 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:06.855210 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:06.855228 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:06.857568 (+ 2340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:06.860669 (+ 3101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:06.860673 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:06.860677 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:06.861096 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:06.861533 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:06.861536 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:06.861539 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:06.861685 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:06.862437 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:06.862444 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:06.862609 (+ 165us) client_negotiation.cc:769] Sending connection context
0811 22:19:06.862740 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 22:19:06.863146 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 22:19:06.863595 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:06.850548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60486 (local address 127.29.229.61:45437)
0811 22:19:06.850696 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:06.850700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:06.850714 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:19:06.850752 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:06.850756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:06.850819 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:06.851063 (+ 244us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:06.851652 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.852528 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.853799 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.854259 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.854371 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:06.857775 (+ 3404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:06.857780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:06.857783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:06.857820 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:06.860446 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:06.861261 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:06.861265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:06.861267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:06.861399 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:06.861834 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:06.861838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:06.861840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:06.862277 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:06.862399 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:06.862889 (+ 490us) server_negotiation.cc:299] Negotiation successful
0811 22:19:06.863239 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":78,"server-negotiator.queue_time_us":52}
I20260811 22:19:06.869058 10197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: Bootstrap starting.
I20260811 22:19:06.872586 10197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:06.873788 10197 log.cc:824] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:06.876677 10197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: No bootstrap required, opened a new log
I20260811 22:19:06.884800 10197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } }
I20260811 22:19:06.885407 10197 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:06.885535 10197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ae333e2d30e4a2e90c526cfbdfc1b79, State: Initialized, Role: FOLLOWER
I20260811 22:19:06.886502 10197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } }
I20260811 22:19:06.887750 10208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } } }
I20260811 22:19:06.888120 10208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:19:06.888995 10197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:19:06.898927 10225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:19:06.899107 10225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:19:06.902963 10093 master_runner.cc:428] Detected that this master 6ae333e2d30e4a2e90c526cfbdfc1b79 is joining an existing cluster
I20260811 22:19:06.903223 10093 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.229.61:45437
I20260811 22:19:06.931561 10206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:06.908761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57674 (local address 127.29.229.62:43683)
0811 22:19:06.908951 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:06.908956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:06.909737 (+ 781us) server_negotiation.cc:407] Connection header received
0811 22:19:06.910071 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:06.910081 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:06.910199 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:06.910285 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:06.912410 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.913758 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.916256 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.917144 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.917270 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:06.921746 (+ 4476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:06.921753 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:06.921757 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:06.921806 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:06.926885 (+ 5079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:06.928189 (+ 1304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:06.928197 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:06.928200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:06.928374 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:06.928921 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:06.928928 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:06.928932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:06.929309 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:06.929740 (+ 431us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:06.930542 (+ 802us) server_negotiation.cc:299] Negotiation successful
0811 22:19:06.931168 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:19:06.933205 10233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:06.908741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.0.0.1:57674)
0811 22:19:06.909617 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:06.909641 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:06.909875 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:06.910744 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:06.910751 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:06.911643 (+ 892us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:06.912211 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:06.912229 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.913981 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.913990 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:06.916059 (+ 2069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:06.916079 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.917662 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.917672 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:06.917942 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:06.919119 (+ 1177us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:06.919155 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:06.921477 (+ 2322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:06.927420 (+ 5943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:06.927424 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:06.927428 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:06.928028 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:06.928563 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:06.928568 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:06.928572 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:06.928769 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:06.929873 (+ 1104us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:06.929878 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:06.930130 (+ 252us) client_negotiation.cc:769] Sending connection context
0811 22:19:06.932206 (+ 2076us) client_negotiation.cc:240] Negotiation successful
0811 22:19:06.932638 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"spinlock_wait_cycles":8960,"thread_start_us":484,"threads_started":1}
I20260811 22:19:06.935846 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:06.910225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60500 (local address 127.29.229.61:45437)
0811 22:19:06.910608 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:06.910617 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:06.920031 (+ 9414us) server_negotiation.cc:407] Connection header received
0811 22:19:06.920233 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:06.920239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:06.920319 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:06.920421 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:06.922006 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.923174 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.925385 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.926100 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.926214 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:06.928800 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:06.928809 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:06.928817 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:06.928854 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:06.932680 (+ 3826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:06.933543 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:06.933547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:06.933550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:06.933752 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:06.934168 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:06.934172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:06.934175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:06.934540 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:06.934686 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:06.935210 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 22:19:06.935461 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215}
I20260811 22:19:06.936874 10234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:06.909809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45437 (local address 127.0.0.1:60500)
0811 22:19:06.919913 (+ 10104us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:06.919929 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:06.920098 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:06.920638 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:06.920644 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:06.921351 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:06.921727 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:06.921739 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.923448 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.923453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:06.925201 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:06.925225 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.926553 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.926560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:06.926768 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:06.927443 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:06.927467 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:06.928633 (+ 1166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:06.932827 (+ 4194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:06.932832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:06.932837 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:06.933422 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:06.933880 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:06.933884 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:06.933887 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:06.934057 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:06.934754 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:06.934759 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:06.934913 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 22:19:06.936094 (+ 1181us) client_negotiation.cc:240] Negotiation successful
0811 22:19:06.936403 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9981,"thread_start_us":9640,"threads_started":1}
WARNING: no policy specified for oryx/127.29.229.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.229.60@KRBTEST.COM" created.
Aug 11 22:19:06 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486744, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.62@KRBTEST.COM for oryx/127.29.229.61@KRBTEST.COM
I20260811 22:19:06.959738 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:06.942046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60508 (local address 127.29.229.61:45437)
0811 22:19:06.942240 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:06.942248 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:06.942480 (+ 232us) server_negotiation.cc:407] Connection header received
0811 22:19:06.942948 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:06.942954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:06.943077 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:06.943193 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:06.944997 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.946332 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.948384 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.949250 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.949403 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:06.953831 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:06.953836 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:06.953840 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:06.953873 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:06.956840 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:06.957566 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:06.957573 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:06.957597 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:06.957734 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:06.958155 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:06.958161 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:06.958164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:06.958443 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:06.958613 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:06.959124 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 22:19:06.959359 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:19:06.959800 10239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:06.941665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45437 (local address 127.0.0.1:60508)
0811 22:19:06.942340 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:06.942384 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:06.942625 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:06.943515 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:06.943524 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:06.944448 (+ 924us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:06.944797 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:06.944811 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.946509 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.946515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:06.948199 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:06.948216 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:06.949439 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:06.949446 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:06.949664 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:06.950491 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:06.950518 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:06.953616 (+ 3098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:06.956999 (+ 3383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:06.957013 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:06.957021 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:06.957435 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:06.957861 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:06.957867 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:06.957871 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:06.958038 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:06.958592 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:06.958601 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:06.958800 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 22:19:06.959015 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 22:19:06.959351 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":579,"threads_started":1}
I20260811 22:19:06.964046 9310 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.229.61:45437: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79"
member_type: NON_VOTER
last_known_addr {
host: "127.29.229.61"
port: 45437
}
attrs {
promote: true
}
}
dest_uuid: "fc323e8e063549859af13dc2f68c4f2f"
cas_config_opid_index: -1
I20260811 22:19:06.965517 9310 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: NON_VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: true } }
I20260811 22:19:06.977085 10157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:06.975666 10244 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6ae333e2d30e4a2e90c526cfbdfc1b79 (127.29.229.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: NON_VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: true } } }
I20260811 22:19:06.981144 10157 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 1 FOLLOWER]: Refusing update from remote peer fc323e8e063549859af13dc2f68c4f2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.229.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.60.keytab.
Entry for principal oryx/127.29.229.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.60.keytab.
I20260811 22:19:06.982440 10245 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: NON_VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:06.984436 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.60:37361
--webserver_interface=127.29.229.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.60.keytab
--principal=oryx/127.29.229.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:43683,127.29.229.61:45437,127.29.229.60:37361
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:19:06.989514 10244 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.229.61:45437
I20260811 22:19:06.992080 10093 master.cc:537] Master@127.29.229.61:45437 shutting down...
I20260811 22:19:06.993268 10245 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: NON_VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: true } } }
I20260811 22:19:06.993888 10245 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:07.020120 10157 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6ae333e2d30e4a2e90c526cfbdfc1b79 (127.29.229.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: NON_VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: true } } }
I20260811 22:19:07.026865 10259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: NON_VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: true } } }
I20260811 22:19:07.027377 10259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:19:07.028065 10255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: NON_VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: true } } }
I20260811 22:19:07.028673 10255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:19:07.028831 10208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc323e8e063549859af13dc2f68c4f2f. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: NON_VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: true } } }
I20260811 22:19:07.029062 10208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:19:07.047420 10093 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 1 LEARNER]: Raft consensus shutting down.
I20260811 22:19:07.048034 10093 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 1 LEARNER]: Raft consensus is shut down!
I20260811 22:19:07.048224 10093 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: stopping tablet replica
I20260811 22:19:07.058744 10093 master.cc:552] Master@127.29.229.61:45437 shutdown complete.
I20260811 22:19:07.059049 10093 master_runner.cc:305] Clearing existing system tablet
I20260811 22:19:07.060724 10093 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:07.060870 10093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:07.065335 10093 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:19:07.067489 10261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:07.067883 10093 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:07.068186 10093 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:19:07.068286 10093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/wal
uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79"
format_stamp: "Formatted at 2026-08-11 22:19:06 on dist-test-slave-jq4t"
server_key: "b9698c452e0aef98c67ddcec0da5553b"
server_key_iv: "88693342d6e7fbabe4bed6bf9d8fad1b"
server_key_version: "encryptionkey@0"
I20260811 22:19:07.073642 10093 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:07.081717 10093 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 22:19:07.081924 10093 log.cc:1196] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 22:19:07.082463 10093 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: Deleting consensus metadata
I20260811 22:19:07.082739 10093 master_runner.cc:315] Copying system tablet from 127.29.229.62:43683
I20260811 22:19:07.093130 10093 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: tablet copy: Beginning tablet copy session from remote peer at address 127.29.229.62:43683
I20260811 22:19:07.113518 10206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.094366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57680 (local address 127.29.229.62:43683)
0811 22:19:07.094591 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:07.094597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:07.097140 (+ 2543us) server_negotiation.cc:407] Connection header received
0811 22:19:07.097371 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:07.097379 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:07.097483 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:07.097605 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:07.098725 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.099935 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.102206 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.102935 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.103078 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.106382 (+ 3304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:07.106390 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:07.106395 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:07.106439 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:07.110186 (+ 3747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.111107 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.111113 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.111117 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.111282 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.111733 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.111738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.111742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.112142 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:07.112307 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:07.112870 (+ 563us) server_negotiation.cc:299] Negotiation successful
0811 22:19:07.113126 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 22:19:07.113678 10267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.094300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.0.0.1:57680)
0811 22:19:07.096990 (+ 2690us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:07.097012 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:07.097191 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:07.097662 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:07.097668 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:07.098230 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:07.098552 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.098565 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.100126 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.100132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.101992 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.102010 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.103329 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.103342 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.103914 (+ 572us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.104900 (+ 986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:07.104931 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:07.106126 (+ 1195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:07.110352 (+ 4226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:07.110357 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:07.110361 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:07.110954 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:07.111410 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:07.111415 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:07.111419 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:07.111596 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:07.112287 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:07.112293 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:07.112526 (+ 233us) client_negotiation.cc:769] Sending connection context
0811 22:19:07.112729 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 22:19:07.113156 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2514,"thread_start_us":2372,"threads_started":1}
I20260811 22:19:07.114835 9349 tablet_copy_service.cc:140] P fc323e8e063549859af13dc2f68c4f2f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6ae333e2d30e4a2e90c526cfbdfc1b79 ({username='oryx', principal='oryx/127.29.229.61@KRBTEST.COM'} at 127.0.0.1:57680)
I20260811 22:19:07.115296 9349 tablet_copy_service.cc:161] P fc323e8e063549859af13dc2f68c4f2f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6ae333e2d30e4a2e90c526cfbdfc1b79 at {username='oryx', principal='oryx/127.29.229.61@KRBTEST.COM'} at 127.0.0.1:57680: session id = 6ae333e2d30e4a2e90c526cfbdfc1b79-00000000000000000000000000000000
I20260811 22:19:07.120543 9349 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 22:19:07.123517 10093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 22:19:07.131815 10093 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: tablet copy: Starting download of 0 data blocks...
I20260811 22:19:07.132308 10093 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: tablet copy: Starting download of 1 WAL segments...
I20260811 22:19:07.136674 10093 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 22:19:07.142381 9349 tablet_copy_service.cc:342] P fc323e8e063549859af13dc2f68c4f2f: Request end of tablet copy session 6ae333e2d30e4a2e90c526cfbdfc1b79-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.229.61@KRBTEST.COM'} at 127.0.0.1:57680
I20260811 22:19:07.142791 9349 tablet_copy_service.cc:433] P fc323e8e063549859af13dc2f68c4f2f: ending tablet copy session 6ae333e2d30e4a2e90c526cfbdfc1b79-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6ae333e2d30e4a2e90c526cfbdfc1b79
I20260811 22:19:07.158224 10093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:07.161298 10268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:07.165668 10271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:07.172526 10275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:07.173278 10093 server_base.cc:1034] running on GCE node
I20260811 22:19:07.173722 10093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:07.174450 10093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:07.175858 10093 hybrid_clock.cc:648] HybridClock initialized: now 1786486747175761 us; error 45 us; skew 500 ppm
I20260811 22:19:07.176122 10093 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:07 dist-test-slave-jq4t krb5kdc[9237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486747, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:07.178714 10093 init.cc:377] Logged in from keytab as oryx/127.29.229.61@KRBTEST.COM (short username oryx)
I20260811 22:19:07.181779 10093 webserver.cc:460] Webserver started at http://127.29.229.61:45105/ using document root <none> and password file <none>
I20260811 22:19:07.182235 10093 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:07.182331 10093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:07.185986 10093 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 22:19:07.188241 10284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:07.188732 10093 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 22:19:07.189070 10093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/wal
uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79"
format_stamp: "Formatted at 2026-08-11 22:19:06 on dist-test-slave-jq4t"
server_key: "b9698c452e0aef98c67ddcec0da5553b"
server_key_iv: "88693342d6e7fbabe4bed6bf9d8fad1b"
server_key_version: "encryptionkey@0"
I20260811 22:19:07.189395 10093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:07.231716 10086 heartbeater.cc:503] Master 127.29.229.62:43683 was elected leader, sending a full tablet report...
I20260811 22:19:07.246325 10093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:07.250955 10093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:07.282514 10093 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.61:45437
I20260811 22:19:07.282611 10365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.61:45437 every 8 connection(s)
I20260811 22:19:07.283901 10093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-1/data/info.pb
I20260811 22:19:07.291029 10367 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:19:07.295627 10367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: Bootstrap starting.
I20260811 22:19:07.312915 10367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:19:07.313557 10367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: Bootstrap complete.
I20260811 22:19:07.314754 10367 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: NON_VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: true } }
I20260811 22:19:07.315068 10367 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6ae333e2d30e4a2e90c526cfbdfc1b79, State: Initialized, Role: LEARNER
I20260811 22:19:07.315483 10367 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: NON_VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: true } }
I20260811 22:19:07.316529 10370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: NON_VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: true } } }
I20260811 22:19:07.316861 10367 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:19:07.317011 10370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:19:07.342213 10388 catalog_manager.cc:1259] Loaded cluster ID: fdc9af6459954353a14b059d59b80b61
I20260811 22:19:07.342510 10388 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: loading cluster ID for follower catalog manager: success
I20260811 22:19:07.346817 10388 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: acquiring CA information for follower catalog manager: success
I20260811 22:19:07.350026 10388 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:19:07.427825 10244 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: NON_VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: true } } }
I20260811 22:19:07.428187 10244 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:07.442883 10239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.428225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45437 (local address 127.0.0.1:60522)
0811 22:19:07.428691 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:07.428714 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:07.428870 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:07.434358 (+ 5488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:07.434365 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:07.434408 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:07.434654 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.434668 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.437274 (+ 2606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.437280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.439429 (+ 2149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.439449 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.441886 (+ 2437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.441895 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.442192 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.442261 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 22:19:07.442384 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 22:19:07.442555 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316}
I20260811 22:19:07.443295 10390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.429153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60522 (local address 127.29.229.61:45437)
0811 22:19:07.433753 (+ 4600us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:07.433761 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:07.433807 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:19:07.433902 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:07.433908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:07.434063 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:07.434247 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:07.434864 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.437091 (+ 2227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.439673 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.441501 (+ 1828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.441872 (+ 371us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.442138 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:07.442466 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 22:19:07.442639 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4430,"thread_start_us":4151,"threads_started":1}
I20260811 22:19:07.444578 10370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc323e8e063549859af13dc2f68c4f2f. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: NON_VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: true } } }
I20260811 22:19:07.444859 10370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:19:07.446161 10247 raft_consensus.cc:1050] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f: attempting to promote NON_VOTER 6ae333e2d30e4a2e90c526cfbdfc1b79 to VOTER
I20260811 22:19:07.447207 10247 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } }
I20260811 22:19:07.449550 10326 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 1 LEARNER]: Refusing update from remote peer fc323e8e063549859af13dc2f68c4f2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 22:19:07.450263 10247 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260811 22:19:07.451771 10253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:07.452353 10253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:07.452574 10253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 22:19:07.458374 10244 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6ae333e2d30e4a2e90c526cfbdfc1b79 (127.29.229.61) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } }
I20260811 22:19:07.459182 10326 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6ae333e2d30e4a2e90c526cfbdfc1b79 (127.29.229.61) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } }
W20260811 22:19:07.462627 10253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
I20260811 22:19:07.462550 10247 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } }
W20260811 22:19:07.462801 10253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:07.462926 10253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
I20260811 22:19:07.462930 10247 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: This master's current role is: LEADER
W20260811 22:19:07.463045 10253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:19:07.463119 10253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:19:07.462596 10244 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } }
I20260811 22:19:07.463310 10244 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:07.464700 10370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } }
I20260811 22:19:07.464949 10370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:19:07.474488 10253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:43683,127.29.229.61:45437,127.29.229.60:37361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.60.keytab
--principal=oryx/127.29.229.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.60:37361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.229.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:07.475929 10253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:07.477407 10253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:07.491910 10406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:07.492435 10402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:07.492728 10408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:07.492937 10253 server_base.cc:1034] running on GCE node
I20260811 22:19:07.494021 10253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:07.495610 10253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:07.496861 10253 hybrid_clock.cc:648] HybridClock initialized: now 1786486747496797 us; error 50 us; skew 500 ppm
I20260811 22:19:07.497236 10253 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:07 dist-test-slave-jq4t krb5kdc[9237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486747, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:07.500597 10253 init.cc:377] Logged in from keytab as oryx/127.29.229.60@KRBTEST.COM (short username oryx)
I20260811 22:19:07.502326 10253 webserver.cc:460] Webserver started at http://127.29.229.60:41599/ using document root <none> and password file <none>
I20260811 22:19:07.503178 10253 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:07.503396 10253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:07.503803 10253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:07.506991 10253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/data/instance:
uuid: "1a26024e37064a939ea8231ab26595d4"
format_stamp: "Formatted at 2026-08-11 22:19:07 on dist-test-slave-jq4t"
server_key: "d5906bd577e72280a42fda8af4c62cd2"
server_key_iv: "499294508ebdba637b1bb0d607e103ce"
server_key_version: "encryptionkey@0"
I20260811 22:19:07.507815 10253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/wal/instance:
uuid: "1a26024e37064a939ea8231ab26595d4"
format_stamp: "Formatted at 2026-08-11 22:19:07 on dist-test-slave-jq4t"
server_key: "d5906bd577e72280a42fda8af4c62cd2"
server_key_iv: "499294508ebdba637b1bb0d607e103ce"
server_key_version: "encryptionkey@0"
I20260811 22:19:07.513306 10253 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 22:19:07.517350 10420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:07.518937 10253 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:19:07.519141 10253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/wal
uuid: "1a26024e37064a939ea8231ab26595d4"
format_stamp: "Formatted at 2026-08-11 22:19:07 on dist-test-slave-jq4t"
server_key: "d5906bd577e72280a42fda8af4c62cd2"
server_key_iv: "499294508ebdba637b1bb0d607e103ce"
server_key_version: "encryptionkey@0"
I20260811 22:19:07.519363 10253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:07.535684 10253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:07.540386 10253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:07.540729 10253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:07.560986 10253 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.60:37361
I20260811 22:19:07.561033 10503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.60:37361 every 8 connection(s)
I20260811 22:19:07.562813 10253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/data/info.pb
I20260811 22:19:07.565387 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 10253
I20260811 22:19:07.567207 10506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:07.575829 10506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:19:07 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.229.60@KRBTEST.COM
Aug 11 22:19:07 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486747, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.60@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM
I20260811 22:19:07.592919 10511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.567624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34918 (local address 127.29.229.60:37361)
0811 22:19:07.568612 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:07.568634 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:07.568682 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:19:07.570448 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:07.570471 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:07.570891 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:07.571331 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:07.573024 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.574557 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.575877 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.576679 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.576892 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.582619 (+ 5727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:07.582626 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:07.582637 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:07.582675 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:07.587710 (+ 5035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.588519 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.588531 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.588542 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.588749 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.589212 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.589216 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.589218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.590003 (+ 785us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:07.590259 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:07.590836 (+ 577us) server_negotiation.cc:299] Negotiation successful
0811 22:19:07.591327 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":371,"threads_started":1}
I20260811 22:19:07.596977 10206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.578768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57684 (local address 127.29.229.62:43683)
0811 22:19:07.579015 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:07.579025 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:07.579930 (+ 905us) server_negotiation.cc:407] Connection header received
0811 22:19:07.580140 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:07.580144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:07.580214 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:07.580304 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:07.581712 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.582914 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.585448 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.585976 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.586091 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.590422 (+ 4331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:07.590427 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:07.590430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:07.590464 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:07.593978 (+ 3514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.594907 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.594911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.594913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.595049 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.595473 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.595476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.595477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.595769 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:07.595945 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:07.596456 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 22:19:07.596740 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 22:19:07.597077 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.578726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.0.0.1:57684)
0811 22:19:07.579728 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:07.579786 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:07.580002 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:07.580409 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:07.580419 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:07.581193 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:07.581543 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.581554 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.583075 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.583083 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.585273 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.585288 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.586217 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.586224 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.586452 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.587200 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:07.587236 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:07.590212 (+ 2976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:07.594113 (+ 3901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:07.594123 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:07.594128 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:07.594761 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:07.595165 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:07.595169 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:07.595173 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:07.595358 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:07.595907 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:07.595933 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:07.596150 (+ 217us) client_negotiation.cc:769] Sending connection context
0811 22:19:07.596318 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 22:19:07.596675 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":453,"threads_started":1}
I20260811 22:19:07.598533 10506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:19:07 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486747, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.60@KRBTEST.COM for oryx/127.29.229.61@KRBTEST.COM
I20260811 22:19:07.613098 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.599368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45437 (local address 127.0.0.1:60526)
0811 22:19:07.599587 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:07.599610 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:07.599722 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:07.600338 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:07.600344 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:07.600994 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:07.601317 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.601330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.602425 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.602429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.603819 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.603834 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.604652 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.604657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.604792 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.605422 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:07.605440 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:07.607608 (+ 2168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:07.610815 (+ 3207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:07.610819 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:07.610821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:07.611262 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:07.611623 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:07.611626 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:07.611628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:07.611758 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:07.612258 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:07.612262 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:07.612392 (+ 130us) client_negotiation.cc:769] Sending connection context
0811 22:19:07.612605 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 22:19:07.612846 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 22:19:07.613122 10390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.599644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60526 (local address 127.29.229.61:45437)
0811 22:19:07.599875 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:07.599880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:07.599898 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:19:07.599954 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:07.599958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:07.600029 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:07.600122 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:07.601489 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.602269 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.603988 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.604408 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.604612 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.607842 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:07.607847 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:07.607850 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:07.607886 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:07.610652 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.611379 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.611383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.611385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.611524 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.611867 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.611870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.611872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.612147 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:07.612261 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:07.612708 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 22:19:07.612902 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 22:19:07.614336 10506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:19:07 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486747, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.60@KRBTEST.COM for oryx/127.29.229.60@KRBTEST.COM
I20260811 22:19:07.627272 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.614975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:37361 (local address 127.0.0.1:34922)
0811 22:19:07.615172 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:07.615193 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:07.615303 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:07.615592 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:07.615595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:07.615928 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:07.616065 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.616073 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.617128 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.617132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.618073 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.618087 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.619030 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.619034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.619193 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.619790 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:07.619807 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:07.622125 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:07.625128 (+ 3003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:07.625132 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:07.625134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:07.625483 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:07.625819 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:07.625822 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:07.625824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:07.625970 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:07.626517 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:07.626520 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:07.626652 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 22:19:07.626762 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 22:19:07.626959 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 22:19:07.627502 10511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.615229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34922 (local address 127.29.229.60:37361)
0811 22:19:07.615370 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:07.615375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:07.615390 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:19:07.615427 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:07.615430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:07.615479 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:07.615552 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:07.616211 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.616998 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.618225 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.618861 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.618998 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.622293 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:07.622298 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:07.622301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:07.622337 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:07.625006 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.625593 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.625596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.625598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.625715 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.626092 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.626096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.626098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.626393 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:07.626514 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:07.626855 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 22:19:07.627117 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":50}
I20260811 22:19:07.632371 10506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: Bootstrap starting.
I20260811 22:19:07.635805 10506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:07.636961 10506 log.cc:824] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:07.640134 10506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: No bootstrap required, opened a new log
I20260811 22:19:07.647867 10506 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } }
I20260811 22:19:07.648231 10506 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:07.648365 10506 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a26024e37064a939ea8231ab26595d4, State: Initialized, Role: FOLLOWER
I20260811 22:19:07.648962 10506 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } }
I20260811 22:19:07.649923 10516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } } }
I20260811 22:19:07.650244 10516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:19:07.650934 10506 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:19:07.651242 10516 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:07.651561 10516 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } }
I20260811 22:19:07.653205 10516 leader_election.cc:288] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc323e8e063549859af13dc2f68c4f2f (127.29.229.62:43683), 6ae333e2d30e4a2e90c526cfbdfc1b79 (127.29.229.61:45437)
I20260811 22:19:07.653941 9322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a26024e37064a939ea8231ab26595d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc323e8e063549859af13dc2f68c4f2f" is_pre_election: true
I20260811 22:19:07.654112 10326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a26024e37064a939ea8231ab26595d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" is_pre_election: true
I20260811 22:19:07.655094 10423 leader_election.cc:302] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a26024e37064a939ea8231ab26595d4; no voters: 6ae333e2d30e4a2e90c526cfbdfc1b79, fc323e8e063549859af13dc2f68c4f2f
I20260811 22:19:07.655509 10516 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 22:19:07.664215 10530 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:19:07.664469 10530 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:19:07.668025 10253 master_runner.cc:428] Detected that this master 1a26024e37064a939ea8231ab26595d4 is joining an existing cluster
I20260811 22:19:07.668172 10253 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.229.60:37361
I20260811 22:19:07.691740 10206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.672392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57698 (local address 127.29.229.62:43683)
0811 22:19:07.672647 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:07.672654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:07.673363 (+ 709us) server_negotiation.cc:407] Connection header received
0811 22:19:07.673657 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:07.673663 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:07.673746 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:07.673835 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:07.675870 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.677131 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.679294 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.680006 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.680107 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.683818 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:07.683826 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:07.683831 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:07.683870 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:07.688028 (+ 4158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.688956 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.688964 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.688968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.689127 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.689645 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.689649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.689652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.690080 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:07.690285 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:07.691106 (+ 821us) server_negotiation.cc:299] Negotiation successful
0811 22:19:07.691357 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 22:19:07.691790 10390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.673531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60536 (local address 127.29.229.61:45437)
0811 22:19:07.673714 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:07.673720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:07.674341 (+ 621us) server_negotiation.cc:407] Connection header received
0811 22:19:07.674647 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:07.674656 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:07.674759 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:07.674844 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:07.676866 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.678250 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.680992 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.681709 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.681853 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.684714 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:07.684722 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:07.684728 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:07.684776 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:07.688416 (+ 3640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.689313 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.689318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.689321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.689470 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.690047 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.690053 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.690056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.690440 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:07.690578 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:07.691211 (+ 633us) server_negotiation.cc:299] Negotiation successful
0811 22:19:07.691443 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:19:07.692983 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.673396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45437 (local address 127.0.0.1:60536)
0811 22:19:07.674235 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:07.674251 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:07.674448 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:07.675119 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:07.675127 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:07.676149 (+ 1022us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:07.676634 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.676651 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.678527 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.678535 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.680820 (+ 2285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.680839 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.682271 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.682284 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.682473 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.683157 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:07.683186 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:07.684399 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:07.688595 (+ 4196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:07.688599 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:07.688603 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:07.689171 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:07.689627 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:07.689631 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:07.689634 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:07.689881 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:07.690664 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:07.690668 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:07.690923 (+ 255us) client_negotiation.cc:769] Sending connection context
0811 22:19:07.691998 (+ 1075us) client_negotiation.cc:240] Negotiation successful
0811 22:19:07.692337 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"mutex_wait_us":1,"spinlock_wait_cycles":66176,"thread_start_us":474,"threads_started":1}
I20260811 22:19:07.693974 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.672443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.0.0.1:57698)
0811 22:19:07.673254 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:07.673274 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:07.673486 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:07.674074 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:07.674078 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:07.675208 (+ 1130us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:07.675655 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.675673 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.677336 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.677341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.679126 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.679141 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.680413 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.680420 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.681155 (+ 735us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.681920 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:07.681949 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:07.683588 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:07.688225 (+ 4637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:07.688230 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:07.688233 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:07.688810 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:07.689291 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:07.689295 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:07.689298 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:07.689505 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:07.690376 (+ 871us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:07.690383 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:07.690658 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 22:19:07.692563 (+ 1905us) client_negotiation.cc:240] Negotiation successful
0811 22:19:07.692920 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":464,"thread_start_us":398,"threads_started":1}
I20260811 22:19:07.698904 10545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.674222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:37361 (local address 127.0.0.1:34932)
0811 22:19:07.680986 (+ 6764us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:07.681004 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:07.681151 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:07.681576 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:07.681581 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:07.682383 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:07.682823 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.682838 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.685358 (+ 2520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.685364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.687206 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.687223 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.688809 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.688816 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.689025 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.689668 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:07.689691 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:07.691048 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:07.695697 (+ 4649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:07.695702 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:07.695706 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:07.696253 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:07.696762 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:07.696768 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:07.696771 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:07.696957 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:07.697736 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:07.697741 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:07.697924 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 22:19:07.698114 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 22:19:07.698437 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6632,"thread_start_us":6467,"threads_started":1}
I20260811 22:19:07.698974 10511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.674541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34932 (local address 127.29.229.60:37361)
0811 22:19:07.674724 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:07.674730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:07.681074 (+ 6344us) server_negotiation.cc:407] Connection header received
0811 22:19:07.681261 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:07.681266 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:07.681329 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:07.681396 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:07.682984 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.684139 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.687373 (+ 3234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.688143 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.688251 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.691240 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:07.691246 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:07.691251 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:07.691286 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:07.695533 (+ 4247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.696405 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.696411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.696414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.696611 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.697091 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.697096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.697101 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.697507 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:07.697654 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:07.698249 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 22:19:07.698594 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":106,"server-negotiator.queue_time_us":71}
I20260811 22:19:07.699762 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 9416
Aug 11 22:19:07 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486744, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.62@KRBTEST.COM for oryx/127.29.229.60@KRBTEST.COM
Aug 11 22:19:07 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.229.1@KRBTEST.COM
I20260811 22:19:07.716856 10239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.699396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:37361 (local address 127.0.0.1:34938)
0811 22:19:07.699625 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:07.699647 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:07.699765 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:07.700677 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:07.700683 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:07.701174 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:07.701421 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.701435 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.703526 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.703533 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.705005 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.705024 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.706081 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.706087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.706261 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.707218 (+ 957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:07.707247 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:07.709789 (+ 2542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:07.714118 (+ 4329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:07.714124 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:07.714128 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:07.714573 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:07.714972 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:07.714983 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:07.714987 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:07.715109 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:07.715798 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:07.715804 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:07.715972 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 22:19:07.716129 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 22:19:07.716426 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 22:19:07.717017 10511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.699573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34938 (local address 127.29.229.60:37361)
0811 22:19:07.700207 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:07.700217 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:07.700235 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:19:07.700338 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:07.700345 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:07.700456 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:07.701964 (+ 1508us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:07.702163 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.703348 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.705182 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.705881 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.706576 (+ 695us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.710008 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:07.710015 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:07.710022 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:07.710060 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:07.713947 (+ 3887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.714705 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.714710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.714714 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.714842 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.715225 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.715231 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.715238 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.715661 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:07.715821 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:07.716223 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 22:19:07.716439 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":407,"threads_started":1}
I20260811 22:19:07.718959 9310 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.229.60:37361: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1a26024e37064a939ea8231ab26595d4"
member_type: NON_VOTER
last_known_addr {
host: "127.29.229.60"
port: 37361
}
attrs {
promote: true
}
}
dest_uuid: "fc323e8e063549859af13dc2f68c4f2f"
cas_config_opid_index: 6
I20260811 22:19:07.719810 9310 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: NON_VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: true } }
I20260811 22:19:07.722494 10326 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 1 FOLLOWER]: Refusing update from remote peer fc323e8e063549859af13dc2f68c4f2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 22:19:07.722563 10469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:07.723256 10247 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 22:19:07.724089 10548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.700988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34258 (local address 127.29.229.1:45275)
0811 22:19:07.701634 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:07.701641 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:07.701753 (+ 112us) server_negotiation.cc:407] Connection header received
0811 22:19:07.701935 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:07.701941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:07.702146 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:07.702345 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:07.703760 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.705140 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.708343 (+ 3203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.709357 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.710118 (+ 761us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.714474 (+ 4356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:07.714490 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:07.714510 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:07.714553 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:07.719228 (+ 4675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.720395 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.720405 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.720408 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.720753 (+ 345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.721247 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.721253 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.721255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.721731 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:07.723214 (+ 1483us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:07.723393 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 22:19:07.723686 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":331,"threads_started":1}
I20260811 22:19:07.725508 9621 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:07.726071 10469 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 1 FOLLOWER]: Refusing update from remote peer fc323e8e063549859af13dc2f68c4f2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 22:19:07.726949 10245 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 1a26024e37064a939ea8231ab26595d4 (127.29.229.60) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: NON_VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: true } } }
I20260811 22:19:07.727594 10326 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 1a26024e37064a939ea8231ab26595d4 (127.29.229.60) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: NON_VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: true } } }
I20260811 22:19:07.727137 10247 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: NON_VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:07.731794 10370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: NON_VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: true } } }
I20260811 22:19:07.732100 10370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:19:07.733609 10245 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.229.60:37361
I20260811 22:19:07.734506 10253 master.cc:537] Master@127.29.229.60:37361 shutting down...
I20260811 22:19:07.735864 10247 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: NON_VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: true } } }
I20260811 22:19:07.736302 10244 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: NON_VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: true } } }
I20260811 22:19:07.736742 10244 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:07.737033 10247 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:07.746331 10469 raft_consensus.cc:858] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: NON_VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } }
W20260811 22:19:07.746551 10469 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: NON_VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } }
I20260811 22:19:07.777350 10469 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6ae333e2d30e4a2e90c526cfbdfc1b79 (127.29.229.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: NON_VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: true } } }
I20260811 22:19:07.782052 10561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: NON_VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: true } } }
I20260811 22:19:07.782455 10555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: NON_VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: true } } }
I20260811 22:19:07.782806 10555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 22:19:07.783164 10516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc323e8e063549859af13dc2f68c4f2f. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: NON_VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: true } } }
I20260811 22:19:07.783392 10516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 22:19:07.784998 10247 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: NON_VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: true } } }
I20260811 22:19:07.785308 10247 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:07.782747 10561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: This master's current role is: NON_PARTICIPANT
W20260811 22:19:07.786180 9287 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260811 22:19:07.787348 10253 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260811 22:19:07.788007 10253 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260811 22:19:07.788229 10253 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: stopping tablet replica
I20260811 22:19:07.799474 10253 master.cc:552] Master@127.29.229.60:37361 shutdown complete.
I20260811 22:19:07.799695 10253 master_runner.cc:305] Clearing existing system tablet
I20260811 22:19:07.800354 10253 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:07.800639 10253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260811 22:19:07.802839 9287 consensus_peers.cc:487] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f -> Peer 1a26024e37064a939ea8231ab26595d4 (127.29.229.60:37361): Couldn't send request to peer 1a26024e37064a939ea8231ab26595d4. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260811 22:19:07.804385 10253 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 22:19:07.806102 10564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:07.806486 10253 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:07.806809 10253 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:19:07.806912 10253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/wal
uuid: "1a26024e37064a939ea8231ab26595d4"
format_stamp: "Formatted at 2026-08-11 22:19:07 on dist-test-slave-jq4t"
server_key: "d5906bd577e72280a42fda8af4c62cd2"
server_key_iv: "499294508ebdba637b1bb0d607e103ce"
server_key_version: "encryptionkey@0"
I20260811 22:19:07.810292 10253 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:07.817499 10253 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 22:19:07.817731 10253 log.cc:1196] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 22:19:07.818657 10253 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: Deleting consensus metadata
I20260811 22:19:07.818946 10253 master_runner.cc:315] Copying system tablet from 127.29.229.62:43683
I20260811 22:19:07.822467 10253 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: tablet copy: Beginning tablet copy session from remote peer at address 127.29.229.62:43683
I20260811 22:19:07.835604 10206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.823305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57700 (local address 127.29.229.62:43683)
0811 22:19:07.823562 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:07.823571 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:07.824131 (+ 560us) server_negotiation.cc:407] Connection header received
0811 22:19:07.824361 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:07.824367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:07.824459 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:07.824598 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:07.825461 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.826184 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.827398 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.827827 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.827975 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.829914 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:07.829920 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:07.829927 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:07.829970 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:07.832835 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.833534 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.833542 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.833545 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.833681 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:07.834090 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:07.834096 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:07.834099 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:07.834396 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:07.834561 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:07.834997 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 22:19:07.835234 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 22:19:07.835672 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:07.823305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.0.0.1:57700)
0811 22:19:07.824022 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:07.824038 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:07.824213 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:07.824626 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:07.824631 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:07.825056 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:07.825303 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.825315 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.826331 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.826336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.827254 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:07.827265 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:07.827990 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:07.827996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:07.828193 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:07.828893 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:07.828914 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:07.829695 (+ 781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:07.833019 (+ 3324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:07.833025 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:07.833029 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:07.833403 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:07.833805 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:07.833810 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:07.833814 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:07.833976 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:07.834521 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:07.834527 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:07.834725 (+ 198us) client_negotiation.cc:769] Sending connection context
0811 22:19:07.834882 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 22:19:07.835227 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":452,"threads_started":1}
I20260811 22:19:07.836735 9349 tablet_copy_service.cc:140] P fc323e8e063549859af13dc2f68c4f2f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1a26024e37064a939ea8231ab26595d4 ({username='oryx', principal='oryx/127.29.229.60@KRBTEST.COM'} at 127.0.0.1:57700)
I20260811 22:19:07.836959 9349 tablet_copy_service.cc:161] P fc323e8e063549859af13dc2f68c4f2f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1a26024e37064a939ea8231ab26595d4 at {username='oryx', principal='oryx/127.29.229.60@KRBTEST.COM'} at 127.0.0.1:57700: session id = 1a26024e37064a939ea8231ab26595d4-00000000000000000000000000000000
I20260811 22:19:07.839458 9349 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 22:19:07.841413 10253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 22:19:07.846424 10253 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: tablet copy: Starting download of 0 data blocks...
I20260811 22:19:07.846774 10253 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: tablet copy: Starting download of 1 WAL segments...
I20260811 22:19:07.849167 10253 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 22:19:07.853408 9349 tablet_copy_service.cc:342] P fc323e8e063549859af13dc2f68c4f2f: Request end of tablet copy session 1a26024e37064a939ea8231ab26595d4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.229.60@KRBTEST.COM'} at 127.0.0.1:57700
I20260811 22:19:07.853574 9349 tablet_copy_service.cc:433] P fc323e8e063549859af13dc2f68c4f2f: ending tablet copy session 1a26024e37064a939ea8231ab26595d4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1a26024e37064a939ea8231ab26595d4
I20260811 22:19:07.865914 10253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:07.868741 10575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:07.879410 10576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:07.879895 10579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:07.880658 10253 server_base.cc:1034] running on GCE node
I20260811 22:19:07.881026 10253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:07.881745 10253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:07.883061 10253 hybrid_clock.cc:648] HybridClock initialized: now 1786486747883011 us; error 38 us; skew 500 ppm
I20260811 22:19:07.883446 10253 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:07 dist-test-slave-jq4t krb5kdc[9237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486747, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:07.885805 10253 init.cc:377] Logged in from keytab as oryx/127.29.229.60@KRBTEST.COM (short username oryx)
I20260811 22:19:07.898499 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 9416
I20260811 22:19:07.942196 10253 webserver.cc:460] Webserver started at http://127.29.229.60:38459/ using document root <none> and password file <none>
I20260811 22:19:07.942775 10253 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:07.942979 10253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:07.943141 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:45275
--local_ip_for_outbound_sockets=127.29.229.1
--tserver_master_addrs=127.29.229.62:43683
--webserver_port=34097
--webserver_interface=127.29.229.1
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.1.keytab
--principal=oryx/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.229.62:43683,127.29.229.61:45437
--tserver_master_addrs=127.29.229.62:43683,127.29.229.61:45437,127.29.229.60:37361 with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:19:07.965306 10253 fs_manager.cc:709] Time spent opening directory manager: real 0.020s user 0.007s sys 0.012s
I20260811 22:19:07.974073 10597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:07.974687 10253 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.007s
I20260811 22:19:07.974893 10253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/wal
uuid: "1a26024e37064a939ea8231ab26595d4"
format_stamp: "Formatted at 2026-08-11 22:19:07 on dist-test-slave-jq4t"
server_key: "d5906bd577e72280a42fda8af4c62cd2"
server_key_iv: "499294508ebdba637b1bb0d607e103ce"
server_key_version: "encryptionkey@0"
I20260811 22:19:07.975234 10253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:08.013808 10253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:08.018541 10253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:08.255424 10253 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.60:37361
I20260811 22:19:08.255445 10680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.60:37361 every 8 connection(s)
I20260811 22:19:08.256794 10253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/master-2/data/info.pb
I20260811 22:19:08.262531 10681 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:19:08.266975 10681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: Bootstrap starting.
I20260811 22:19:08.284920 10681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:19:08.285714 10681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: Bootstrap complete.
I20260811 22:19:08.287153 10681 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: NON_VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: true } }
I20260811 22:19:08.287601 10681 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1a26024e37064a939ea8231ab26595d4, State: Initialized, Role: LEARNER
I20260811 22:19:08.287962 10681 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: NON_VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: true } }
I20260811 22:19:08.295027 10684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: NON_VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: true } } }
I20260811 22:19:08.296188 10681 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:19:08.297435 10684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:19:08.311259 10245 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: NON_VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: true } } }
I20260811 22:19:08.311585 10245 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:08.361797 10710 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 22:19:08.369267 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:08.311697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34946 (local address 127.29.229.60:37361)
0811 22:19:08.325675 (+ 13978us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:08.325685 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:08.346312 (+ 20627us) server_negotiation.cc:407] Connection header received
0811 22:19:08.346601 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:08.346607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:08.346735 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:08.346887 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:08.348190 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:08.349786 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:08.353234 (+ 3448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:08.353996 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:08.354102 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:08.359712 (+ 5610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:08.359725 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:08.359731 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:08.359800 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:08.365435 (+ 5635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:08.366637 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:08.366643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:08.366647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:08.366821 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:08.367382 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:08.367388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:08.367392 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:08.367777 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:08.367949 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:08.368575 (+ 626us) server_negotiation.cc:299] Negotiation successful
0811 22:19:08.368814 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13795,"thread_start_us":14006,"threads_started":1}
I20260811 22:19:08.369746 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:08.311469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:37361 (local address 127.0.0.1:34946)
0811 22:19:08.346134 (+ 34665us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:08.346167 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:08.346425 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:08.347218 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:08.347223 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:08.347783 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:08.348008 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:08.348020 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:08.350054 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:08.350061 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:08.351831 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:08.351859 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:08.354440 (+ 2581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:08.354447 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:08.354643 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:08.356218 (+ 1575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:08.356246 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:08.358025 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:08.365673 (+ 7648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:08.365682 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:08.365690 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:08.366465 (+ 775us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:08.366984 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:08.366992 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:08.366998 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:08.367234 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:08.367942 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:08.367950 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:08.368183 (+ 233us) client_negotiation.cc:769] Sending connection context
0811 22:19:08.368411 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 22:19:08.368920 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34462,"thread_start_us":35110,"threads_started":1}
I20260811 22:19:08.370896 10644 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260811 22:19:08.371382 10684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc323e8e063549859af13dc2f68c4f2f. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: NON_VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: true } } }
I20260811 22:19:08.371661 10684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:19:08.372254 10699 raft_consensus.cc:1050] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f: attempting to promote NON_VOTER 1a26024e37064a939ea8231ab26595d4 to VOTER
I20260811 22:19:08.373137 10699 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: false } }
I20260811 22:19:08.375321 10326 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 1 FOLLOWER]: Refusing update from remote peer fc323e8e063549859af13dc2f68c4f2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 22:19:08.375380 10644 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 1 LEARNER]: Refusing update from remote peer fc323e8e063549859af13dc2f68c4f2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 22:19:08.376005 10245 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 22:19:08.376296 10698 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 22:19:08.379758 10245 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 1a26024e37064a939ea8231ab26595d4 (127.29.229.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: false } } }
I20260811 22:19:08.384016 10245 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: false } } }
I20260811 22:19:08.384718 10245 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:08.384443 10699 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: false } } }
I20260811 22:19:08.384943 10699 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:08.384215 10326 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 1a26024e37064a939ea8231ab26595d4 (127.29.229.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: false } } }
I20260811 22:19:08.386142 10699 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: false } } }
I20260811 22:19:08.386356 10699 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:08.387154 10644 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 1a26024e37064a939ea8231ab26595d4 (127.29.229.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: false } } }
I20260811 22:19:08.391702 10684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: false } } }
I20260811 22:19:08.391983 10684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:19:08.392982 10713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fc323e8e063549859af13dc2f68c4f2f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: false } } }
I20260811 22:19:08.393299 10713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 22:19:08.525532 10595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:08.525858 10595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:08.525975 10595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:08.536198 10595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:08.536348 10595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:08.536599 10595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:19:08.548192 10595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.1.keytab
--principal=oryx/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:45275
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=34097
--tserver_master_addrs=127.29.229.62:43683,127.29.229.61:45437,127.29.229.60:37361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:08.549711 10595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:08.551169 10595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:08.566213 10719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:08.566138 10722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:08.566365 10720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:08.570158 10595 server_base.cc:1034] running on GCE node
I20260811 22:19:08.570959 10595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:08.572067 10595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:08.573314 10595 hybrid_clock.cc:648] HybridClock initialized: now 1786486748573247 us; error 61 us; skew 500 ppm
I20260811 22:19:08.573745 10595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:08 dist-test-slave-jq4t krb5kdc[9237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486748, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:08.577469 10595 init.cc:377] Logged in from keytab as oryx/127.29.229.1@KRBTEST.COM (short username oryx)
I20260811 22:19:08.579334 10595 webserver.cc:460] Webserver started at http://127.29.229.1:34097/ using document root <none> and password file <none>
I20260811 22:19:08.580109 10595 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:08.580282 10595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:08.586690 10595 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:19:08.589955 10736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:08.591935 10595 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:19:08.592159 10595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "be34d3c2b4c5470caf70aac065cc5d31"
format_stamp: "Formatted at 2026-08-11 22:19:04 on dist-test-slave-jq4t"
server_key: "b1a0a9414656780a5e6297e3f3241449"
server_key_iv: "acbdfe9cd38e695274cb06d2f8ee679f"
server_key_version: "encryptionkey@0"
I20260811 22:19:08.593442 10595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:08.611740 10595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:08.616567 10595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:08.616950 10595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:08.618193 10595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:08.619874 10595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:08.620000 10595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:08.620379 10595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:08.620530 10595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:08.661346 10595 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:45275
I20260811 22:19:08.661410 10927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:45275 every 8 connection(s)
I20260811 22:19:08.662832 10595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:19:08.666009 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 10595
I20260811 22:19:08.666433 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 9644
Aug 11 22:19:08 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486748, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.60@KRBTEST.COM
Aug 11 22:19:08 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.229.2@KRBTEST.COM
Aug 11 22:19:08 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486748, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.61@KRBTEST.COM
Aug 11 22:19:08 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486748, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM
I20260811 22:19:08.707878 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:08.679399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:46653 (local address 127.29.229.60:37361)
0811 22:19:08.679637 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:08.679647 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:08.681206 (+ 1559us) server_negotiation.cc:407] Connection header received
0811 22:19:08.686433 (+ 5227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:08.686443 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:08.686570 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:08.686667 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:08.690873 (+ 4206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:08.691758 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:08.693516 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:08.694305 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:08.694450 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:08.699627 (+ 5177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:08.699633 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:08.699638 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:08.699683 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:08.703358 (+ 3675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:08.704305 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:08.704313 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:08.704316 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:08.704501 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:08.705131 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:08.705136 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:08.705140 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:08.705591 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:08.705747 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:08.707187 (+ 1440us) server_negotiation.cc:299] Negotiation successful
0811 22:19:08.707445 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 22:19:08.710742 10937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:08.679813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:37361 (local address 127.29.229.1:46653)
0811 22:19:08.681100 (+ 1287us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:08.681120 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:08.685918 (+ 4798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:08.687946 (+ 2028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:08.687964 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:08.689362 (+ 1398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:08.690615 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:08.690645 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:08.691939 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:08.691942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:08.693349 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:08.693363 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:08.694791 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:08.694802 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:08.695128 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:08.695892 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:08.695924 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:08.699346 (+ 3422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:08.703491 (+ 4145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:08.703501 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:08.703522 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:08.704166 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:08.704657 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:08.704664 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:08.704672 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:08.704934 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:08.706187 (+ 1253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:08.706197 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:08.706682 (+ 485us) client_negotiation.cc:769] Sending connection context
0811 22:19:08.708217 (+ 1535us) client_negotiation.cc:240] Negotiation successful
0811 22:19:08.708880 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1152,"thread_start_us":964,"threads_started":1}
I20260811 22:19:08.712815 10928 heartbeater.cc:347] Connected to a master server at 127.29.229.60:37361
I20260811 22:19:08.713220 10928 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:08.713935 10932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:08.666115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:53809 (local address 127.29.229.62:43683)
0811 22:19:08.694676 (+ 28561us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:08.694683 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:08.694703 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:19:08.694906 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:08.694911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:08.694995 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:08.695106 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:08.696733 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:08.699205 (+ 2472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:08.701369 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:08.702098 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:08.702228 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:08.706731 (+ 4503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:08.706738 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:08.706743 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:08.706786 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:08.710682 (+ 3896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:08.711532 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:08.711539 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:08.711542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:08.711715 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:08.712215 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:08.712220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:08.712223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:08.712604 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:08.712753 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:08.713280 (+ 527us) server_negotiation.cc:299] Negotiation successful
0811 22:19:08.713503 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28381,"thread_start_us":28565,"threads_started":1}
I20260811 22:19:08.713967 10933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:08.666821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.1:53809)
0811 22:19:08.678167 (+ 11346us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:08.678239 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:08.694742 (+ 16503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:08.695340 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:08.695346 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:08.695998 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:08.696350 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:08.696366 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:08.699431 (+ 3065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:08.699438 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:08.701176 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:08.701193 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:08.702488 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:08.702495 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:08.702717 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:08.703419 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:08.703448 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:08.706455 (+ 3007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:08.710877 (+ 4422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:08.710882 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:08.710885 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:08.711367 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:08.711856 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:08.711861 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:08.711867 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:08.712061 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:08.712825 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:08.712830 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:08.712995 (+ 165us) client_negotiation.cc:769] Sending connection context
0811 22:19:08.713156 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 22:19:08.713500 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10875,"mutex_wait_us":3,"thread_start_us":10587,"threads_started":1}
I20260811 22:19:08.714365 10928 heartbeater.cc:511] Master 127.29.229.60:37361 requested a full tablet report, sending...
I20260811 22:19:08.714632 10935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:08.667331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40020 (local address 127.29.229.2:41123)
0811 22:19:08.682422 (+ 15091us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:08.682439 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:08.682499 (+ 60us) server_negotiation.cc:407] Connection header received
0811 22:19:08.682733 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:08.682763 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:08.683202 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:08.686816 (+ 3614us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:08.687221 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:08.689397 (+ 2176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:08.693396 (+ 3999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:08.694496 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:08.702091 (+ 7595us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:08.702410 (+ 319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:08.702427 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:08.702448 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:08.702494 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:08.708610 (+ 6116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:08.709662 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:08.709671 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:08.709674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:08.709860 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:08.711165 (+ 1305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:08.711170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:08.711173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:08.711682 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:08.713179 (+ 1497us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:08.713442 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 22:19:08.713761 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14720,"thread_start_us":14447,"threads_started":1}
I20260811 22:19:08.716049 9848 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:08.717109 10625 ts_manager.cc:194] Registered new tserver with Master: be34d3c2b4c5470caf70aac065cc5d31 (127.29.229.1:45275)
I20260811 22:19:08.717669 10929 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43683
I20260811 22:19:08.717832 10929 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:08.718379 10929 heartbeater.cc:511] Master 127.29.229.62:43683 requested a full tablet report, sending...
I20260811 22:19:08.719000 10944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:08.687866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45437 (local address 127.29.229.1:50917)
0811 22:19:08.690707 (+ 2841us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:08.690728 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:08.690870 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:08.694923 (+ 4053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:08.694928 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:08.695688 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:08.696261 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:08.696280 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:08.698034 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:08.698040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:08.699645 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:08.699659 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:08.700978 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:08.700987 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:08.701246 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:08.701962 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:08.701987 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:08.705042 (+ 3055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:08.709685 (+ 4643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:08.709691 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:08.709695 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:08.710899 (+ 1204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:08.711405 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:08.711411 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:08.711414 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:08.711586 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:08.715999 (+ 4413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:08.716004 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:08.716187 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 22:19:08.716360 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 22:19:08.717309 (+ 949us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2705,"mutex_wait_us":53,"thread_start_us":2517,"threads_started":1}
I20260811 22:19:08.719837 9310 ts_manager.cc:194] Re-registered known tserver with Master: be34d3c2b4c5470caf70aac065cc5d31 (127.29.229.1:45275)
I20260811 22:19:08.721413 9310 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:53809
I20260811 22:19:08.723124 10941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:08.681442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:50917 (local address 127.29.229.61:45437)
0811 22:19:08.694407 (+ 12965us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:08.694413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:08.694433 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:19:08.694494 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:08.694499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:08.694588 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:08.696436 (+ 1848us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:08.696659 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:08.697847 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:08.699829 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:08.700539 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:08.700667 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:08.705242 (+ 4575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:08.705247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:08.705250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:08.705284 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:08.709469 (+ 4185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:08.711061 (+ 1592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:08.711066 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:08.711069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:08.711235 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:08.712177 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:08.712182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:08.712185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:08.712595 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:08.717877 (+ 5282us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:08.722163 (+ 4286us) server_negotiation.cc:299] Negotiation successful
0811 22:19:08.722541 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12788,"thread_start_us":12418,"threads_started":1}
I20260811 22:19:08.724193 10930 heartbeater.cc:347] Connected to a master server at 127.29.229.61:45437
I20260811 22:19:08.724367 10930 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:08.725044 10930 heartbeater.cc:511] Master 127.29.229.61:45437 requested a full tablet report, sending...
I20260811 22:19:08.727067 10314 ts_manager.cc:194] Registered new tserver with Master: be34d3c2b4c5470caf70aac065cc5d31 (127.29.229.1:45275)
I20260811 22:19:08.811872 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 9644
I20260811 22:19:08.837229 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:41123
--local_ip_for_outbound_sockets=127.29.229.2
--tserver_master_addrs=127.29.229.62:43683
--webserver_port=33195
--webserver_interface=127.29.229.2
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.2.keytab
--principal=oryx/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.229.62:43683,127.29.229.61:45437
--tserver_master_addrs=127.29.229.62:43683,127.29.229.61:45437,127.29.229.60:37361 with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:09.208760 10946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:09.209160 10946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:09.209304 10946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:09.218189 10946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:09.218353 10946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:09.218523 10946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:19:09.229179 10946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.2.keytab
--principal=oryx/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:41123
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=33195
--tserver_master_addrs=127.29.229.62:43683,127.29.229.61:45437,127.29.229.60:37361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:09.230536 10946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:09.231870 10946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:09.246263 10954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:09.246292 10955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:09.247104 10962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:09.248315 10946 server_base.cc:1034] running on GCE node
I20260811 22:19:09.249060 10946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:09.250178 10946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:09.251423 10946 hybrid_clock.cc:648] HybridClock initialized: now 1786486749251372 us; error 42 us; skew 500 ppm
I20260811 22:19:09.251794 10946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:09 dist-test-slave-jq4t krb5kdc[9237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486749, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:09.256124 10946 init.cc:377] Logged in from keytab as oryx/127.29.229.2@KRBTEST.COM (short username oryx)
I20260811 22:19:09.257928 10946 webserver.cc:460] Webserver started at http://127.29.229.2:33195/ using document root <none> and password file <none>
I20260811 22:19:09.258623 10946 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:09.258747 10946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:09.264766 10946 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:19:09.268636 10971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:09.270287 10946 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:19:09.270443 10946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "655348f7beb645b98a09af862a80c1ee"
format_stamp: "Formatted at 2026-08-11 22:19:05 on dist-test-slave-jq4t"
server_key: "a84e269e4772a74fb592df63e34d5771"
server_key_iv: "e14a4d12b5f3d637d82c7594d16cfdd5"
server_key_version: "encryptionkey@0"
I20260811 22:19:09.271564 10946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:09.300244 10946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:09.304962 10946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:09.305231 10946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:09.306257 10946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:09.307777 10946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:09.307870 10946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:09.307943 10946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:09.308009 10946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:09.348775 10946 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:41123
I20260811 22:19:09.348876 11144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:41123 every 8 connection(s)
I20260811 22:19:09.350284 10946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:19:09.358582 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 10946
I20260811 22:19:09.358922 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 9869
I20260811 22:19:09.364467 10710 catalog_manager.cc:1259] Loaded cluster ID: fdc9af6459954353a14b059d59b80b61
I20260811 22:19:09.364622 10710 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: loading cluster ID for follower catalog manager: success
I20260811 22:19:09.367992 10710 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: acquiring CA information for follower catalog manager: success
I20260811 22:19:09.370527 10710 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 11 22:19:09 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486749, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.60@KRBTEST.COM
Aug 11 22:19:09 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486749, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM
Aug 11 22:19:09 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486749, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.61@KRBTEST.COM
I20260811 22:19:09.386190 11158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:09.355342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:40705 (local address 127.29.229.60:37361)
0811 22:19:09.356407 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:09.356416 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:09.356441 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:19:09.358559 (+ 2118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:09.358566 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:09.358657 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:09.358773 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:09.361644 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:09.363233 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:09.366415 (+ 3182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:09.367364 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:09.367532 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:09.375047 (+ 7515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:09.375054 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:09.375058 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:09.375099 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:09.379066 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:09.380111 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:09.380117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:09.380122 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:09.380280 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:09.380850 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:09.380854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:09.380858 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:09.381285 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:09.381437 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:09.384682 (+ 3245us) server_negotiation.cc:299] Negotiation successful
0811 22:19:09.384987 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":877,"spinlock_wait_cycles":11136,"thread_start_us":854,"threads_started":1}
I20260811 22:19:09.386516 11157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:09.355176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:37361 (local address 127.29.229.2:40705)
0811 22:19:09.355858 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:09.355876 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:09.358110 (+ 2234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:09.359213 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:09.359224 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:09.360281 (+ 1057us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:09.361379 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:09.361409 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:09.363497 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:09.363505 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:09.366185 (+ 2680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:09.366206 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:09.367920 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:09.367929 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:09.368317 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:09.369636 (+ 1319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:09.369669 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:09.374789 (+ 5120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:09.379234 (+ 4445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:09.379246 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:09.379263 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:09.379945 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:09.380426 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:09.380432 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:09.380436 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:09.380706 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:09.381516 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:09.381525 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:09.381996 (+ 471us) client_negotiation.cc:769] Sending connection context
0811 22:19:09.382350 (+ 354us) client_negotiation.cc:240] Negotiation successful
0811 22:19:09.382957 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":1,"thread_start_us":341,"threads_started":1}
Aug 11 22:19:09 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.229.3@KRBTEST.COM
I20260811 22:19:09.388671 11154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:09.353298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:37935 (local address 127.29.229.61:45437)
0811 22:19:09.353855 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:09.353860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:09.354862 (+ 1002us) server_negotiation.cc:407] Connection header received
0811 22:19:09.358746 (+ 3884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:09.358755 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:09.358875 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:09.360052 (+ 1177us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:09.362823 (+ 2771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:09.363932 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:09.371330 (+ 7398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:09.371993 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:09.372153 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:09.378721 (+ 6568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:09.378727 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:09.378733 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:09.378784 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:09.383728 (+ 4944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:09.384741 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:09.384747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:09.384750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:09.384903 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:09.385419 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:09.385425 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:09.385429 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:09.386003 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:09.386169 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:09.388031 (+ 1862us) server_negotiation.cc:299] Negotiation successful
0811 22:19:09.388255 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":292,"threads_started":1}
I20260811 22:19:09.388943 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:09.353975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45437 (local address 127.29.229.2:37935)
0811 22:19:09.354710 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:09.354746 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:09.358092 (+ 3346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:09.359215 (+ 1123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:09.359228 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:09.362369 (+ 3141us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:09.362640 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:09.362651 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:09.364127 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:09.364132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:09.371160 (+ 7028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:09.371176 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:09.372455 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:09.372461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:09.372782 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:09.374661 (+ 1879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:09.374683 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:09.378464 (+ 3781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:09.383939 (+ 5475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:09.383944 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:09.383947 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:09.384603 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:09.385088 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:09.385093 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:09.385097 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:09.385285 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:09.387480 (+ 2195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:09.387484 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:09.387701 (+ 217us) client_negotiation.cc:769] Sending connection context
0811 22:19:09.387876 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 22:19:09.388188 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"mutex_wait_us":4973,"thread_start_us":293,"threads_started":1}
I20260811 22:19:09.390352 11148 heartbeater.cc:347] Connected to a master server at 127.29.229.60:37361
I20260811 22:19:09.390427 11152 heartbeater.cc:347] Connected to a master server at 127.29.229.61:45437
I20260811 22:19:09.390718 11152 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:09.390725 11148 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:09.391770 11161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:09.355921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.2:56469)
0811 22:19:09.356951 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:09.356966 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:09.358477 (+ 1511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:09.360974 (+ 2497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:09.360979 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:09.361616 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:09.361965 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:09.361980 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:09.365645 (+ 3665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:09.365650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:09.367219 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:09.367234 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:09.368334 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:09.368341 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:09.368621 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:09.369413 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:09.369444 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:09.374016 (+ 4572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:09.386542 (+ 12526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:09.386553 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:09.386556 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:09.387348 (+ 792us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:09.389670 (+ 2322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:09.389677 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:09.389682 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:09.389919 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:09.390644 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:09.390650 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:09.390843 (+ 193us) client_negotiation.cc:769] Sending connection context
0811 22:19:09.391033 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 22:19:09.391362 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":920,"thread_start_us":767,"threads_started":1}
I20260811 22:19:09.391872 11148 heartbeater.cc:511] Master 127.29.229.60:37361 requested a full tablet report, sending...
I20260811 22:19:09.392412 11162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:09.356065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:56469 (local address 127.29.229.62:43683)
0811 22:19:09.360639 (+ 4574us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:09.360647 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:09.360668 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:19:09.360733 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:09.360737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:09.360814 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:09.360952 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:09.364262 (+ 3310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:09.365451 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:09.367413 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:09.368018 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:09.368958 (+ 940us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:09.374280 (+ 5322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:09.374286 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:09.374290 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:09.374331 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:09.386074 (+ 11743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:09.389290 (+ 3216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:09.389298 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:09.389304 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:09.389481 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:09.390056 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:09.390063 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:09.390067 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:09.390513 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:09.390666 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:09.391185 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 22:19:09.391443 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4421,"thread_start_us":4293,"threads_started":1}
I20260811 22:19:09.392611 11152 heartbeater.cc:511] Master 127.29.229.61:45437 requested a full tablet report, sending...
I20260811 22:19:09.393839 10314 ts_manager.cc:194] Registered new tserver with Master: 655348f7beb645b98a09af862a80c1ee (127.29.229.2:41123)
I20260811 22:19:09.394201 10625 ts_manager.cc:194] Registered new tserver with Master: 655348f7beb645b98a09af862a80c1ee (127.29.229.2:41123)
I20260811 22:19:09.394829 11150 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43683
I20260811 22:19:09.394969 11150 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:09.395401 11150 heartbeater.cc:511] Master 127.29.229.62:43683 requested a full tablet report, sending...
I20260811 22:19:09.396564 9310 ts_manager.cc:194] Re-registered known tserver with Master: 655348f7beb645b98a09af862a80c1ee (127.29.229.2:41123)
I20260811 22:19:09.397483 9310 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:56469
I20260811 22:19:09.398988 11166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:09.360330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52188 (local address 127.29.229.3:46419)
0811 22:19:09.373673 (+ 13343us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:09.373680 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:09.373708 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:19:09.373768 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:09.373774 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:09.373978 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:09.374107 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:09.376911 (+ 2804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:09.378569 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:09.380776 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:09.381851 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:09.382020 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:09.389892 (+ 7872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:09.389912 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:09.389931 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:09.390027 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:09.395482 (+ 5455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:09.396333 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:09.396342 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:09.396346 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:09.396577 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:09.397051 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:09.397056 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:09.397060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:09.397406 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:09.397729 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:09.398435 (+ 706us) server_negotiation.cc:299] Negotiation successful
0811 22:19:09.398690 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13138,"thread_start_us":13999,"threads_started":1}
I20260811 22:19:09.400161 10078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:09.489795 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 9869
I20260811 22:19:09.517364 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:46419
--local_ip_for_outbound_sockets=127.29.229.3
--tserver_master_addrs=127.29.229.62:43683
--webserver_port=38879
--webserver_interface=127.29.229.3
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.3.keytab
--principal=oryx/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.229.62:43683,127.29.229.61:45437
--tserver_master_addrs=127.29.229.62:43683,127.29.229.61:45437,127.29.229.60:37361 with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:19:09.725638 10929 heartbeater.cc:503] Master 127.29.229.62:43683 was elected leader, sending a full tablet report...
W20260811 22:19:09.895643 11170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:09.895939 11170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:09.896100 11170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:09.904346 11170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:09.904537 11170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:09.904687 11170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:19:09.915050 11170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.3.keytab
--principal=oryx/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:46419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=38879
--tserver_master_addrs=127.29.229.62:43683,127.29.229.61:45437,127.29.229.60:37361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:09.916448 11170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:09.917910 11170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:09.932691 11181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:09.932998 11176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:09.933162 11184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:09.933553 11170 server_base.cc:1034] running on GCE node
I20260811 22:19:09.934307 11170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:09.935366 11170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:09.936594 11170 hybrid_clock.cc:648] HybridClock initialized: now 1786486749936530 us; error 75 us; skew 500 ppm
I20260811 22:19:09.937029 11170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:09 dist-test-slave-jq4t krb5kdc[9237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486749, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:09.940722 11170 init.cc:377] Logged in from keytab as oryx/127.29.229.3@KRBTEST.COM (short username oryx)
I20260811 22:19:09.942543 11170 webserver.cc:460] Webserver started at http://127.29.229.3:38879/ using document root <none> and password file <none>
I20260811 22:19:09.943567 11170 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:09.943768 11170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:09.950094 11170 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 22:19:09.954066 11196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:09.955881 11170 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260811 22:19:09.956131 11170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "43ca70cae5964345831bcafd0d7178bf"
format_stamp: "Formatted at 2026-08-11 22:19:06 on dist-test-slave-jq4t"
server_key: "94a0af77ba2d9c81b6379716056c4660"
server_key_iv: "a3627404d642a5f3156a31921817cd97"
server_key_version: "encryptionkey@0"
I20260811 22:19:09.957360 11170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:09.986191 11170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:09.991812 11170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:09.992105 11170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:09.993204 11170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:09.994755 11170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:09.994851 11170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:09.994926 11170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:09.994993 11170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:10.036989 11170 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:46419
I20260811 22:19:10.037146 11375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:46419 every 8 connection(s)
I20260811 22:19:10.038661 11170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:19:10.039572 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 11170
Aug 11 22:19:10 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486749, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM
I20260811 22:19:10.089035 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.042879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:55179 (local address 127.29.229.62:43683)
0811 22:19:10.052638 (+ 9759us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:10.052645 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:10.058427 (+ 5782us) server_negotiation.cc:407] Connection header received
0811 22:19:10.058566 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:10.058576 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:10.058742 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:10.058891 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:10.062903 (+ 4012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.064102 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.066956 (+ 2854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.067908 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.068049 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.078321 (+ 10272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:10.078329 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:10.078333 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:10.078394 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:10.083587 (+ 5193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.085916 (+ 2329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.085924 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.085930 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.086111 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.086684 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.086693 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.086699 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.087198 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:10.087365 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:10.088339 (+ 974us) server_negotiation.cc:299] Negotiation successful
0811 22:19:10.088635 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9621,"thread_start_us":9430,"threads_started":1}
I20260811 22:19:10.090533 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.043547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.3:55179)
0811 22:19:10.053588 (+ 10041us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:10.053679 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:10.057624 (+ 3945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:10.059612 (+ 1988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:10.059631 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:10.060924 (+ 1293us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:10.062415 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:10.062451 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.064416 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.064423 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:10.066730 (+ 2307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:10.066752 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.068537 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.068549 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:10.068898 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.072744 (+ 3846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:10.072789 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:10.078022 (+ 5233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:10.083735 (+ 5713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:10.083750 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:10.083772 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:10.084525 (+ 753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:10.086215 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:10.086225 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:10.086229 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:10.086557 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:10.087338 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:10.087347 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:10.087892 (+ 545us) client_negotiation.cc:769] Sending connection context
0811 22:19:10.088235 (+ 343us) client_negotiation.cc:240] Negotiation successful
0811 22:19:10.088869 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9497,"thread_start_us":9155,"threads_started":1}
I20260811 22:19:10.092374 11377 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43683
I20260811 22:19:10.092629 11393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.052633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57706 (local address 127.29.229.62:43683)
0811 22:19:10.059431 (+ 6798us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:10.059436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:10.060379 (+ 943us) server_negotiation.cc:407] Connection header received
0811 22:19:10.060756 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:10.060762 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:10.060867 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:10.060963 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:10.064230 (+ 3267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.065444 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.070209 (+ 4765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.071065 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.071191 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.077203 (+ 6012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:10.077211 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:10.077215 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:10.077262 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:10.085725 (+ 8463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.087876 (+ 2151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.087881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.087884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.088028 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.089955 (+ 1927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.089959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.089963 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.090364 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:10.090908 (+ 544us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:10.091900 (+ 992us) server_negotiation.cc:299] Negotiation successful
0811 22:19:10.092128 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6625,"thread_start_us":6818,"threads_started":1}
I20260811 22:19:10.092700 11377 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:10.093770 11377 heartbeater.cc:511] Master 127.29.229.62:43683 requested a full tablet report, sending...
I20260811 22:19:10.095745 9310 ts_manager.cc:194] Re-registered known tserver with Master: 43ca70cae5964345831bcafd0d7178bf (127.29.229.3:46419)
I20260811 22:19:10.097024 9310 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:55179
Aug 11 22:19:10 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486749, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.60@KRBTEST.COM
I20260811 22:19:10.112705 11396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.060872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60542 (local address 127.29.229.61:45437)
0811 22:19:10.088048 (+ 27176us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:10.088054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:10.088073 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:19:10.088137 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:10.088145 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:10.088255 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:10.088381 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:10.092315 (+ 3934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.093853 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.096776 (+ 2923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.097517 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.097644 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.101985 (+ 4341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:10.101993 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:10.101998 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:10.102057 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:10.107784 (+ 5727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.108695 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.108703 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.108710 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.108881 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.109385 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.109390 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.109393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.109770 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:10.110029 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:10.111683 (+ 1654us) server_negotiation.cc:299] Negotiation successful
0811 22:19:10.111959 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27054,"spinlock_wait_cycles":4992,"thread_start_us":26940,"threads_started":1}
Aug 11 22:19:10 dist-test-slave-jq4t krb5kdc[9237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486749, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.61@KRBTEST.COM
I20260811 22:19:10.115298 11398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.061058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:37361 (local address 127.29.229.3:40033)
0811 22:19:10.079606 (+ 18548us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:10.079628 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:10.079752 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:10.086104 (+ 6352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:10.086113 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:10.087000 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:10.087509 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:10.087526 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.090086 (+ 2560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.090091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:10.092073 (+ 1982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:10.092096 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.095358 (+ 3262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.095369 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:10.095546 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.096266 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:10.096288 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:10.100718 (+ 4430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:10.106694 (+ 5976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:10.106700 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:10.106704 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:10.107277 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:10.107776 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:10.107781 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:10.107784 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:10.107959 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:10.108767 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:10.108775 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:10.110833 (+ 2058us) client_negotiation.cc:769] Sending connection context
0811 22:19:10.110999 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 22:19:10.111355 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11343,"thread_start_us":11397,"threads_started":1}
I20260811 22:19:10.104454 9309 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 22:19:10.116552 11403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.055082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:40033 (local address 127.29.229.60:37361)
0811 22:19:10.085410 (+ 30328us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:10.085420 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:10.085450 (+ 30us) server_negotiation.cc:407] Connection header received
0811 22:19:10.085534 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:10.085543 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:10.085674 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:10.085825 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:10.088615 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.089844 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.094037 (+ 4193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.094795 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.094926 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.100956 (+ 6030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:10.100963 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:10.100968 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:10.101014 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:10.106545 (+ 5531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.107451 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.107459 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.107464 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.107629 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.108102 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.108109 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.108112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.108549 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:10.108688 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:10.115777 (+ 7089us) server_negotiation.cc:299] Negotiation successful
0811 22:19:10.116044 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30141,"thread_start_us":39992,"threads_started":1}
I20260811 22:19:10.117537 11376 heartbeater.cc:347] Connected to a master server at 127.29.229.60:37361
I20260811 22:19:10.117710 11376 heartbeater.cc:464] Registering TS with master...
W20260811 22:19:10.117784 9309 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:19:10.118250 11376 heartbeater.cc:511] Master 127.29.229.60:37361 requested a full tablet report, sending...
I20260811 22:19:10.119742 10625 ts_manager.cc:194] Registered new tserver with Master: 43ca70cae5964345831bcafd0d7178bf (127.29.229.3:46419)
I20260811 22:19:10.124369 11401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.075053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45437 (local address 127.29.229.3:46415)
0811 22:19:10.082295 (+ 7242us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:10.082320 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:10.082485 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:10.104738 (+ 22253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:10.104744 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:10.105425 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:10.105745 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:10.105760 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.108863 (+ 3103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.108867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:10.110512 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:10.110529 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.111583 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.111592 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:10.111754 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.112369 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:10.112403 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:10.115001 (+ 2598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:10.120934 (+ 5933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:10.120942 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:10.120946 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:10.121435 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:10.121885 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:10.121890 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:10.121893 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:10.122047 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:10.123230 (+ 1183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:10.123235 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:10.123437 (+ 202us) client_negotiation.cc:769] Sending connection context
0811 22:19:10.123588 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 22:19:10.123912 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7065,"thread_start_us":6864,"threads_started":1}
I20260811 22:19:10.125044 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.088024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:46415 (local address 127.29.229.61:45437)
0811 22:19:10.104209 (+ 16185us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:10.104215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:10.104238 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:19:10.104289 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:10.104293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:10.104382 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:10.104704 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:10.107289 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.108735 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.110667 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.111393 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.111525 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.115224 (+ 3699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:10.115233 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:10.115239 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:10.115994 (+ 755us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:10.120797 (+ 4803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.121567 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.121575 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.121579 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.121765 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.122170 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.122176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.122179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.122525 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:10.122672 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:10.124284 (+ 1612us) server_negotiation.cc:299] Negotiation successful
0811 22:19:10.124593 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16043,"thread_start_us":16492,"threads_started":1}
I20260811 22:19:10.126223 11395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.078040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34956 (local address 127.29.229.60:37361)
0811 22:19:10.091843 (+ 13803us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:10.091849 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:10.094346 (+ 2497us) server_negotiation.cc:407] Connection header received
0811 22:19:10.094411 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:10.094416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:10.094527 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:10.094632 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:10.100282 (+ 5650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.105105 (+ 4823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.108931 (+ 3826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.109679 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.109803 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.114569 (+ 4766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:10.114576 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:10.114581 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:10.114621 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:10.118741 (+ 4120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.119701 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.119709 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.119713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.119918 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.120387 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.120391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.120394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.120772 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:10.121049 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:10.125614 (+ 4565us) server_negotiation.cc:299] Negotiation successful
0811 22:19:10.125873 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13621,"thread_start_us":7045,"threads_started":1}
I20260811 22:19:10.127207 11379 heartbeater.cc:347] Connected to a master server at 127.29.229.61:45437
I20260811 22:19:10.127389 11379 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:10.127818 11379 heartbeater.cc:511] Master 127.29.229.61:45437 requested a full tablet report, sending...
I20260811 22:19:10.132618 10314 ts_manager.cc:194] Registered new tserver with Master: 43ca70cae5964345831bcafd0d7178bf (127.29.229.3:46419)
I20260811 22:19:10.193710 11429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.146206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34260 (local address 127.29.229.1:45275)
0811 22:19:10.180581 (+ 34375us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:10.180598 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:10.183713 (+ 3115us) server_negotiation.cc:407] Connection header received
0811 22:19:10.184087 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:10.184097 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:10.184424 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:10.184685 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:10.186439 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.187970 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.190785 (+ 2815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.192517 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.192661 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.192943 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:10.193171 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 22:19:10.193362 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34044,"thread_start_us":35006,"threads_started":1}
I20260811 22:19:10.196255 11431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.145877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:45275 (local address 127.0.0.1:34260)
0811 22:19:10.183516 (+ 37639us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:10.183543 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:10.183790 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:10.185500 (+ 1710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:10.185508 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:10.185547 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:10.186155 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:10.186170 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.188138 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.188144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:10.190606 (+ 2462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:10.190621 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.192739 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.192745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:10.192943 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.192983 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 22:19:10.193117 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 22:19:10.193368 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37431,"mutex_wait_us":47,"thread_start_us":43996,"threads_started":1}
I20260811 22:19:10.199214 11427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.146110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:46419 (local address 127.0.0.1:52196)
0811 22:19:10.188871 (+ 42761us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:10.188897 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:10.189084 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:10.190074 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:10.190079 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:10.190121 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:10.190556 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:10.190570 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.192696 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.192701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:10.195805 (+ 3104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:10.195823 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.198178 (+ 2355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.198184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:10.198384 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.198430 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 22:19:10.198528 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 22:19:10.198786 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42562,"mutex_wait_us":4,"spinlock_wait_cycles":3840,"thread_start_us":43324,"threads_started":1}
I20260811 22:19:10.199219 11430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.146558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52196 (local address 127.29.229.3:46419)
0811 22:19:10.184623 (+ 38065us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:10.184635 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:10.189041 (+ 4406us) server_negotiation.cc:407] Connection header received
0811 22:19:10.189290 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:10.189300 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:10.189577 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:10.189778 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:10.190859 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.192534 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.196147 (+ 3613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.197960 (+ 1813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.198110 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.198616 (+ 506us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:10.198700 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 22:19:10.198899 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37785,"thread_start_us":36071,"threads_started":1}
I20260811 22:19:10.199515 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.148448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40024 (local address 127.29.229.2:41123)
0811 22:19:10.181183 (+ 32735us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:10.181196 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:10.189849 (+ 8653us) server_negotiation.cc:407] Connection header received
0811 22:19:10.190431 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:10.190439 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:10.190660 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:10.190866 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:10.191817 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.193072 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.196666 (+ 3594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.198010 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.198140 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.198348 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:10.199002 (+ 654us) server_negotiation.cc:299] Negotiation successful
0811 22:19:10.199200 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32474,"thread_start_us":32997,"threads_started":1}
I20260811 22:19:10.200248 11428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.148160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:41123 (local address 127.0.0.1:40024)
0811 22:19:10.189722 (+ 41562us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:10.189738 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:10.190146 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:10.191175 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:10.191182 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:10.191207 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:10.191644 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:10.191663 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.193245 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.193250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:10.196457 (+ 3207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:10.196476 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.198584 (+ 2108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.198590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:10.198775 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.198806 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 22:19:10.199734 (+ 928us) client_negotiation.cc:240] Negotiation successful
0811 22:19:10.199873 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41433,"spinlock_wait_cycles":11136,"thread_start_us":34908,"threads_started":1}
I20260811 22:19:10.200842 10817 tablet_service.cc:1467] Processing CreateTablet for tablet 97e5d1bff45147aa9a6d300ffb111a52 (DEFAULT_TABLE table=test-table [id=db13f8fa8a8640da85ea38c351e99718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:10.202443 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97e5d1bff45147aa9a6d300ffb111a52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:10.203150 11042 tablet_service.cc:1467] Processing CreateTablet for tablet 97e5d1bff45147aa9a6d300ffb111a52 (DEFAULT_TABLE table=test-table [id=db13f8fa8a8640da85ea38c351e99718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:10.203117 11274 tablet_service.cc:1467] Processing CreateTablet for tablet 97e5d1bff45147aa9a6d300ffb111a52 (DEFAULT_TABLE table=test-table [id=db13f8fa8a8640da85ea38c351e99718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:10.204241 11042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97e5d1bff45147aa9a6d300ffb111a52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:10.204567 11274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97e5d1bff45147aa9a6d300ffb111a52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:10.217823 11433 tablet_bootstrap.cc:492] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee: Bootstrap starting.
I20260811 22:19:10.221446 11437 tablet_bootstrap.cc:492] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf: Bootstrap starting.
I20260811 22:19:10.222795 11433 tablet_bootstrap.cc:654] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:10.224220 11433 log.cc:824] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:10.225312 11437 tablet_bootstrap.cc:654] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:10.225795 11436 tablet_bootstrap.cc:492] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31: Bootstrap starting.
I20260811 22:19:10.226610 11437 log.cc:824] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:10.229458 11436 tablet_bootstrap.cc:654] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:10.229810 11437 tablet_bootstrap.cc:492] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf: No bootstrap required, opened a new log
I20260811 22:19:10.230089 11437 ts_tablet_manager.cc:1397] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 22:19:10.230552 11436 log.cc:824] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:10.232534 11433 tablet_bootstrap.cc:492] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee: No bootstrap required, opened a new log
I20260811 22:19:10.232834 11433 ts_tablet_manager.cc:1397] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.000s
I20260811 22:19:10.235085 11436 tablet_bootstrap.cc:492] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31: No bootstrap required, opened a new log
I20260811 22:19:10.235380 11436 ts_tablet_manager.cc:1397] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260811 22:19:10.238410 11437 raft_consensus.cc:348] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } } peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } }
I20260811 22:19:10.238812 11437 raft_consensus.cc:374] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:10.238914 11437 raft_consensus.cc:729] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43ca70cae5964345831bcafd0d7178bf, State: Initialized, Role: FOLLOWER
I20260811 22:19:10.239562 11437 consensus_queue.cc:260] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } } peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } }
I20260811 22:19:10.241663 11437 ts_tablet_manager.cc:1428] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf: Time spent starting tablet: real 0.011s user 0.012s sys 0.001s
I20260811 22:19:10.241977 11377 heartbeater.cc:503] Master 127.29.229.62:43683 was elected leader, sending a full tablet report...
I20260811 22:19:10.244369 11433 raft_consensus.cc:348] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } } peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } }
I20260811 22:19:10.244879 11433 raft_consensus.cc:374] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:10.244971 11433 raft_consensus.cc:729] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 655348f7beb645b98a09af862a80c1ee, State: Initialized, Role: FOLLOWER
I20260811 22:19:10.245712 11433 consensus_queue.cc:260] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } } peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } }
I20260811 22:19:10.248494 11436 raft_consensus.cc:348] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } } peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } }
I20260811 22:19:10.249110 11436 raft_consensus.cc:374] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:10.249240 11436 raft_consensus.cc:729] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be34d3c2b4c5470caf70aac065cc5d31, State: Initialized, Role: FOLLOWER
I20260811 22:19:10.250054 11436 consensus_queue.cc:260] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } } peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } }
I20260811 22:19:10.252410 11436 ts_tablet_manager.cc:1428] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31: Time spent starting tablet: real 0.017s user 0.012s sys 0.005s
I20260811 22:19:10.253870 11433 ts_tablet_manager.cc:1428] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee: Time spent starting tablet: real 0.021s user 0.015s sys 0.005s
I20260811 22:19:10.255496 11150 heartbeater.cc:503] Master 127.29.229.62:43683 was elected leader, sending a full tablet report...
W20260811 22:19:10.299568 11380 tablet.cc:2369] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:19:10.354853 11153 tablet.cc:2369] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:19:10.432008 11444 raft_consensus.cc:482] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:10.432423 11444 raft_consensus.cc:504] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } } peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } }
I20260811 22:19:10.434435 11444 leader_election.cc:288] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43ca70cae5964345831bcafd0d7178bf (127.29.229.3:46419), 655348f7beb645b98a09af862a80c1ee (127.29.229.2:41123)
W20260811 22:19:10.442103 10931 tablet.cc:2369] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:19:10.443765 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.435031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:40345 (local address 127.29.229.2:41123)
0811 22:19:10.435306 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:10.435310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:10.436762 (+ 1452us) server_negotiation.cc:407] Connection header received
0811 22:19:10.437090 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:10.437096 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:10.437183 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:10.437291 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:10.438079 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.438867 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.441095 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.442456 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.442638 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.442714 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:10.443327 (+ 613us) server_negotiation.cc:299] Negotiation successful
0811 22:19:10.443477 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260811 22:19:10.444419 11430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.434970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:56109 (local address 127.29.229.3:46419)
0811 22:19:10.435215 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:10.435222 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:10.437163 (+ 1941us) server_negotiation.cc:407] Connection header received
0811 22:19:10.437374 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:10.437380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:10.437463 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:10.437584 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:10.438311 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.439183 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.441275 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.442521 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.442648 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.442714 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:10.443773 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0811 22:19:10.444177 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:19:10.444545 11446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.434995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:41123 (local address 127.29.229.1:40345)
0811 22:19:10.436472 (+ 1477us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:10.436558 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:10.436862 (+ 304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:10.437375 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:10.437381 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:10.437416 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:10.437909 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:10.437921 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.439038 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.439044 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:10.440805 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:10.440844 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.442918 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.442928 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:10.443113 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.443156 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 22:19:10.443979 (+ 823us) client_negotiation.cc:240] Negotiation successful
0811 22:19:10.444107 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1072,"mutex_wait_us":31,"thread_start_us":739,"threads_started":1}
I20260811 22:19:10.444602 11450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.435732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:46419 (local address 127.29.229.1:56109)
0811 22:19:10.436975 (+ 1243us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:10.437004 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:10.437191 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:10.437614 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:10.437618 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:10.437640 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:10.438096 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:10.438109 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.439338 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.439342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:10.441076 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:10.441091 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.443410 (+ 2319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.443417 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:10.443575 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.443609 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 22:19:10.443702 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 22:19:10.444056 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"mutex_wait_us":47,"thread_start_us":696,"threads_started":1}
I20260811 22:19:10.445708 11074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e5d1bff45147aa9a6d300ffb111a52" candidate_uuid: "be34d3c2b4c5470caf70aac065cc5d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "655348f7beb645b98a09af862a80c1ee" is_pre_election: true
I20260811 22:19:10.445708 11305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e5d1bff45147aa9a6d300ffb111a52" candidate_uuid: "be34d3c2b4c5470caf70aac065cc5d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43ca70cae5964345831bcafd0d7178bf" is_pre_election: true
I20260811 22:19:10.446345 11074 raft_consensus.cc:2454] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be34d3c2b4c5470caf70aac065cc5d31 in term 0.
I20260811 22:19:10.446336 11305 raft_consensus.cc:2454] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be34d3c2b4c5470caf70aac065cc5d31 in term 0.
I20260811 22:19:10.447082 10742 leader_election.cc:302] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 655348f7beb645b98a09af862a80c1ee, be34d3c2b4c5470caf70aac065cc5d31; no voters:
I20260811 22:19:10.447474 11444 raft_consensus.cc:2781] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:19:10.447630 11444 raft_consensus.cc:482] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:19:10.447748 11444 raft_consensus.cc:3037] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:10.450287 11444 raft_consensus.cc:504] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } } peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } }
I20260811 22:19:10.451023 11444 leader_election.cc:288] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [CANDIDATE]: Term 1 election: Requested vote from peers 43ca70cae5964345831bcafd0d7178bf (127.29.229.3:46419), 655348f7beb645b98a09af862a80c1ee (127.29.229.2:41123)
I20260811 22:19:10.451545 11305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e5d1bff45147aa9a6d300ffb111a52" candidate_uuid: "be34d3c2b4c5470caf70aac065cc5d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43ca70cae5964345831bcafd0d7178bf"
I20260811 22:19:10.451632 11074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e5d1bff45147aa9a6d300ffb111a52" candidate_uuid: "be34d3c2b4c5470caf70aac065cc5d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "655348f7beb645b98a09af862a80c1ee"
I20260811 22:19:10.451768 11305 raft_consensus.cc:3037] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:10.451922 11074 raft_consensus.cc:3037] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:10.454243 11305 raft_consensus.cc:2454] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be34d3c2b4c5470caf70aac065cc5d31 in term 1.
I20260811 22:19:10.454770 10741 leader_election.cc:302] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43ca70cae5964345831bcafd0d7178bf, be34d3c2b4c5470caf70aac065cc5d31; no voters:
I20260811 22:19:10.454828 11074 raft_consensus.cc:2454] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be34d3c2b4c5470caf70aac065cc5d31 in term 1.
I20260811 22:19:10.455137 11444 raft_consensus.cc:2781] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:19:10.455683 11444 raft_consensus.cc:686] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [term 1 LEADER]: Becoming Leader. State: Replica: be34d3c2b4c5470caf70aac065cc5d31, State: Running, Role: LEADER
I20260811 22:19:10.456326 11444 consensus_queue.cc:237] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } } peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } }
I20260811 22:19:10.463792 9309 catalog_manager.cc:5495] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 reported cstate change: term changed from 0 to 1, leader changed from <none> to be34d3c2b4c5470caf70aac065cc5d31 (127.29.229.1). New cstate: current_term: 1 leader_uuid: "be34d3c2b4c5470caf70aac065cc5d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } health_report { overall_health: UNKNOWN } } }
I20260811 22:19:10.509303 11429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.502303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34268 (local address 127.29.229.1:45275)
0811 22:19:10.502494 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:10.502499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:10.502741 (+ 242us) server_negotiation.cc:407] Connection header received
0811 22:19:10.502915 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:10.502919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:10.503025 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:10.503190 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:19:10.503790 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.504790 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.506310 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.506736 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.506838 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.507512 (+ 674us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:19:10.508333 (+ 821us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:19:10.508586 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:10.508897 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 22:19:10.508993 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:19:10.518915 11074 raft_consensus.cc:1261] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee [term 1 FOLLOWER]: Refusing update from remote peer be34d3c2b4c5470caf70aac065cc5d31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:10.518909 11305 raft_consensus.cc:1261] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf [term 1 FOLLOWER]: Refusing update from remote peer be34d3c2b4c5470caf70aac065cc5d31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:10.520241 11456 consensus_queue.cc:1035] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:10.520773 11444 consensus_queue.cc:1035] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:10.556728 9310 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:10.557009 9310 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:10.563970 9310 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc323e8e063549859af13dc2f68c4f2f: Sending DeleteTablet for 3 replicas of tablet 97e5d1bff45147aa9a6d300ffb111a52
I20260811 22:19:10.565413 11274 tablet_service.cc:1514] Processing DeleteTablet for tablet 97e5d1bff45147aa9a6d300ffb111a52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:10 UTC) from {username='oryx', principal='oryx/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:52196
I20260811 22:19:10.565491 11042 tablet_service.cc:1514] Processing DeleteTablet for tablet 97e5d1bff45147aa9a6d300ffb111a52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:10 UTC) from {username='oryx', principal='oryx/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:40024
I20260811 22:19:10.566619 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet 97e5d1bff45147aa9a6d300ffb111a52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:10 UTC) from {username='oryx', principal='oryx/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:34260
I20260811 22:19:10.568308 11475 tablet_replica.cc:330] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee: stopping tablet replica
I20260811 22:19:10.568884 11475 raft_consensus.cc:2229] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:10.569540 11475 raft_consensus.cc:2258] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:10.570565 11479 tablet_replica.cc:330] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31: stopping tablet replica
I20260811 22:19:10.570940 11479 raft_consensus.cc:2229] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:19:10.571586 11479 raft_consensus.cc:2258] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:10.573333 11475 ts_tablet_manager.cc:1905] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:10.580421 11477 tablet_replica.cc:330] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf: stopping tablet replica
I20260811 22:19:10.580394 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 9255
I20260811 22:19:10.581007 11477 raft_consensus.cc:2229] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:10.582105 9362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:10.584910 11479 ts_tablet_manager.cc:1905] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:10.588203 11477 raft_consensus.cc:2258] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:10.590235 11475 ts_tablet_manager.cc:1918] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:10.590456 11475 log.cc:1196] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/97e5d1bff45147aa9a6d300ffb111a52
I20260811 22:19:10.591185 11475 ts_tablet_manager.cc:1939] T 97e5d1bff45147aa9a6d300ffb111a52 P 655348f7beb645b98a09af862a80c1ee: Deleting consensus metadata
I20260811 22:19:10.592259 11477 ts_tablet_manager.cc:1905] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:10.597415 11479 ts_tablet_manager.cc:1918] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:10.597661 11479 log.cc:1196] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/wal/wals/97e5d1bff45147aa9a6d300ffb111a52
I20260811 22:19:10.598659 11479 ts_tablet_manager.cc:1939] T 97e5d1bff45147aa9a6d300ffb111a52 P be34d3c2b4c5470caf70aac065cc5d31: Deleting consensus metadata
I20260811 22:19:10.602160 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.569151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60544 (local address 127.29.229.61:45437)
0811 22:19:10.569360 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:10.569366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:10.578648 (+ 9282us) server_negotiation.cc:407] Connection header received
0811 22:19:10.579888 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:10.579897 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:10.580023 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:10.580127 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:10.582999 (+ 2872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.584536 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.587380 (+ 2844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.588147 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.588268 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.592936 (+ 4668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:10.592944 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:10.592948 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:10.592995 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:10.597777 (+ 4782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.599131 (+ 1354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.599137 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.599141 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.599301 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.599785 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.599790 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.599793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.600207 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:10.600508 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:10.601322 (+ 814us) server_negotiation.cc:299] Negotiation successful
0811 22:19:10.601683 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:19:10.609221 11477 ts_tablet_manager.cc:1918] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:10.609478 11477 log.cc:1196] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/97e5d1bff45147aa9a6d300ffb111a52
I20260811 22:19:10.610105 11477 ts_tablet_manager.cc:1939] T 97e5d1bff45147aa9a6d300ffb111a52 P 43ca70cae5964345831bcafd0d7178bf: Deleting consensus metadata
I20260811 22:19:10.621657 11395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.578824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34958 (local address 127.29.229.60:37361)
0811 22:19:10.579020 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:10.579027 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:10.603986 (+ 24959us) server_negotiation.cc:407] Connection header received
0811 22:19:10.604272 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:10.604277 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:10.604374 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:10.604472 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:10.607756 (+ 3284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.608956 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.611243 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.611903 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.612009 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.615210 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:10.615215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:10.615218 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:10.615256 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:10.618491 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.619202 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.619209 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.619212 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.619349 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.619764 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.619769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.619773 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.620246 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:10.620469 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:10.621045 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 22:19:10.621292 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:19:10.785182 9290 catalog_manager.cc:4841] TS 655348f7beb645b98a09af862a80c1ee (127.29.229.2:41123): tablet 97e5d1bff45147aa9a6d300ffb111a52 (table test-table [id=db13f8fa8a8640da85ea38c351e99718]) successfully deleted
I20260811 22:19:10.785203 9289 catalog_manager.cc:4841] TS 43ca70cae5964345831bcafd0d7178bf (127.29.229.3:46419): tablet 97e5d1bff45147aa9a6d300ffb111a52 (table test-table [id=db13f8fa8a8640da85ea38c351e99718]) successfully deleted
W20260811 22:19:10.785544 9290 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.29.229.2:41123, user_credentials={real_user=slave}} blocked reactor thread for 191317us
I20260811 22:19:10.786291 9287 catalog_manager.cc:4841] TS be34d3c2b4c5470caf70aac065cc5d31 (127.29.229.1:45275): tablet 97e5d1bff45147aa9a6d300ffb111a52 (table test-table [id=db13f8fa8a8640da85ea38c351e99718]) successfully deleted
I20260811 22:19:10.786417 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 9255
I20260811 22:19:10.851557 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.832107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60556 (local address 127.29.229.61:45437)
0811 22:19:10.832318 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:10.832326 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:10.832758 (+ 432us) server_negotiation.cc:407] Connection header received
0811 22:19:10.833068 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:10.833076 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:10.833179 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:10.833273 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:10.835673 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.836769 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.838210 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.838642 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.838789 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.843004 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:10.843012 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:10.843016 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:10.843067 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:10.847110 (+ 4043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.848785 (+ 1675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.848791 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.848795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.848947 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.849373 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.849378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.849382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.849724 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:10.850267 (+ 543us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:10.850527 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 22:19:10.850960 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:19:10.852128 11395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:10.832754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34960 (local address 127.29.229.60:37361)
0811 22:19:10.832944 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:10.832951 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:10.833618 (+ 667us) server_negotiation.cc:407] Connection header received
0811 22:19:10.833757 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:10.833762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:10.833841 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:10.833951 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:10.835997 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.836870 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.838816 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:10.839445 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:10.839571 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:10.843165 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:10.843172 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:10.843179 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:10.843225 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:10.847087 (+ 3862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.848045 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.848050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.848052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.848182 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:10.848697 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:10.848702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:10.848705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:10.849162 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:10.849428 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:10.851361 (+ 1933us) server_negotiation.cc:299] Negotiation successful
0811 22:19:10.851625 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:19:11.543056 11495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:11.542157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.1:38727)
0811 22:19:11.542669 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:11.542823 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":306,"threads_started":1}
W20260811 22:19:11.543637 10929 heartbeater.cc:644] Failed to heartbeat to 127.29.229.62:43683 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
I20260811 22:19:11.548805 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:11.547789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.2:48955)
0811 22:19:11.548257 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:11.548369 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":234,"threads_started":1}
W20260811 22:19:11.549705 11150 heartbeater.cc:644] Failed to heartbeat to 127.29.229.62:43683 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
I20260811 22:19:11.551862 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:11.550977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.3:58047)
0811 22:19:11.551489 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:11.551641 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":287,"threads_started":1}
W20260811 22:19:11.552333 11377 heartbeater.cc:644] Failed to heartbeat to 127.29.229.62:43683 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
I20260811 22:19:12.063820 11499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc323e8e063549859af13dc2f68c4f2f)
I20260811 22:19:12.063838 11500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc323e8e063549859af13dc2f68c4f2f)
I20260811 22:19:12.064101 11499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: false } }
I20260811 22:19:12.064162 11500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: false } }
I20260811 22:19:12.064972 11499 leader_election.cc:288] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc323e8e063549859af13dc2f68c4f2f (127.29.229.62:43683), 6ae333e2d30e4a2e90c526cfbdfc1b79 (127.29.229.61:45437)
I20260811 22:19:12.065871 11500 leader_election.cc:288] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc323e8e063549859af13dc2f68c4f2f (127.29.229.62:43683), 1a26024e37064a939ea8231ab26595d4 (127.29.229.60:37361)
I20260811 22:19:12.066561 11502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.065209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.0.0.1:58074)
0811 22:19:12.065811 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.066138 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":282,"threads_started":1}
W20260811 22:19:12.067029 10600 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
I20260811 22:19:12.068095 11504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.065988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.0.0.1:58078)
0811 22:19:12.067559 (+ 1571us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.067760 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1466,"thread_start_us":1351,"threads_started":1}
W20260811 22:19:12.069483 10286 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
I20260811 22:19:12.070767 11502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.070131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.0.0.1:58084)
0811 22:19:12.070405 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.070531 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260811 22:19:12.071020 10600 leader_election.cc:334] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fc323e8e063549859af13dc2f68c4f2f (127.29.229.62:43683): Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
I20260811 22:19:12.072759 11510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.071986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.0.0.1:58092)
0811 22:19:12.072255 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.072455 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260811 22:19:12.073061 10286 leader_election.cc:334] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fc323e8e063549859af13dc2f68c4f2f (127.29.229.62:43683): Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
I20260811 22:19:12.074977 11503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.065873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45437 (local address 127.0.0.1:60570)
0811 22:19:12.066439 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.066461 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:12.066658 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:12.067761 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:12.067766 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:12.067793 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:12.068556 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:12.068580 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.070223 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.070232 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:12.072697 (+ 2465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:12.072710 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.074049 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.074058 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:12.074263 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:12.074328 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 22:19:12.074454 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 22:19:12.074601 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":455,"threads_started":1}
I20260811 22:19:12.074996 11506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.066192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60570 (local address 127.29.229.61:45437)
0811 22:19:12.067364 (+ 1172us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:12.067372 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:12.067398 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:19:12.067483 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:12.067488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:12.067571 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:12.068104 (+ 533us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:12.068840 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.070061 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.072853 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.073856 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.074029 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:12.074280 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:12.074474 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 22:19:12.074613 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":105,"server-negotiator.queue_time_us":1006,"thread_start_us":811,"threads_started":1}
I20260811 22:19:12.075636 10326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a26024e37064a939ea8231ab26595d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" is_pre_election: true
I20260811 22:19:12.075963 10326 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a26024e37064a939ea8231ab26595d4 in term 1.
I20260811 22:19:12.076644 10600 leader_election.cc:302] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1a26024e37064a939ea8231ab26595d4, 6ae333e2d30e4a2e90c526cfbdfc1b79; no voters: fc323e8e063549859af13dc2f68c4f2f
I20260811 22:19:12.076980 11499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 22:19:12.077117 11499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc323e8e063549859af13dc2f68c4f2f)
I20260811 22:19:12.077280 11499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:19:12.077625 11504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.067843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:37361 (local address 127.0.0.1:34966)
0811 22:19:12.069800 (+ 1957us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.069826 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:12.069992 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:12.071970 (+ 1978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:12.071976 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:12.072005 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:12.072315 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:12.072330 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.073760 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.073766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:12.075225 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:12.075324 (+ 99us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.076805 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.076811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:12.076991 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:12.077027 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:19:12.077165 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 22:19:12.077365 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":3,"thread_start_us":1363,"threads_started":1}
I20260811 22:19:12.077973 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.068598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34966 (local address 127.29.229.60:37361)
0811 22:19:12.071614 (+ 3016us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:12.071620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:12.071643 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:19:12.071707 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:12.071712 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:12.071784 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:12.071913 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:12.072558 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.073623 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.075518 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.076603 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.076767 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:12.077012 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:12.077231 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 22:19:12.077396 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2887,"thread_start_us":2701,"threads_started":1}
I20260811 22:19:12.078758 10644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1a26024e37064a939ea8231ab26595d4" is_pre_election: true
I20260811 22:19:12.080181 11499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: false } }
I20260811 22:19:12.080718 10644 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ae333e2d30e4a2e90c526cfbdfc1b79 in current term 2: Already voted for candidate 1a26024e37064a939ea8231ab26595d4 in this term.
I20260811 22:19:12.080891 11499 leader_election.cc:288] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [CANDIDATE]: Term 2 election: Requested vote from peers fc323e8e063549859af13dc2f68c4f2f (127.29.229.62:43683), 6ae333e2d30e4a2e90c526cfbdfc1b79 (127.29.229.61:45437)
I20260811 22:19:12.081604 11503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.080992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.0.0.1:58096)
0811 22:19:12.081356 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.081462 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250}
I20260811 22:19:12.081576 10326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a26024e37064a939ea8231ab26595d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79"
I20260811 22:19:12.081874 10326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:19:12.081920 10286 leader_election.cc:302] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ae333e2d30e4a2e90c526cfbdfc1b79; no voters: 1a26024e37064a939ea8231ab26595d4, fc323e8e063549859af13dc2f68c4f2f
I20260811 22:19:12.082701 11503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.082254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.0.0.1:58106)
0811 22:19:12.082470 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.082561 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260811 22:19:12.082981 10600 leader_election.cc:334] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer fc323e8e063549859af13dc2f68c4f2f (127.29.229.62:43683): Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
I20260811 22:19:12.084383 10326 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a26024e37064a939ea8231ab26595d4 in term 2.
I20260811 22:19:12.084630 11500 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:19:12.085036 10600 leader_election.cc:302] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1a26024e37064a939ea8231ab26595d4, 6ae333e2d30e4a2e90c526cfbdfc1b79; no voters: fc323e8e063549859af13dc2f68c4f2f
I20260811 22:19:12.085521 11499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:19:12.086246 11499 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [term 2 LEADER]: Becoming Leader. State: Replica: 1a26024e37064a939ea8231ab26595d4, State: Running, Role: LEADER
I20260811 22:19:12.086841 11499 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: false } }
I20260811 22:19:12.089090 11514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a26024e37064a939ea8231ab26595d4. Latest consensus state: current_term: 2 leader_uuid: "1a26024e37064a939ea8231ab26595d4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: false } } }
I20260811 22:19:12.089587 11514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:12.090327 11517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:19:12.094630 11517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:19:12.095566 11517 catalog_manager.cc:1259] Loaded cluster ID: fdc9af6459954353a14b059d59b80b61
I20260811 22:19:12.095705 11517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:19:12.097230 11517 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:19:12.098191 11517 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: Loaded TSK: 0
I20260811 22:19:12.099251 11517 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:19:12.116031 10625 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:19:12.123674 10625 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:19:12.128898 10326 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [term 2 FOLLOWER]: Refusing update from remote peer 1a26024e37064a939ea8231ab26595d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 22:19:12.129060 11503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.128334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.0.0.1:58124)
0811 22:19:12.128574 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.128659 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 22:19:12.129734 11514 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 22:19:12.130240 11503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.129770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.0.0.1:58134)
0811 22:19:12.130021 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.130113 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 22:19:12.130468 10600 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 -> Peer fc323e8e063549859af13dc2f68c4f2f (127.29.229.62:43683): Couldn't send request to peer fc323e8e063549859af13dc2f68c4f2f. Status: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 22:19:12.132459 11500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a26024e37064a939ea8231ab26595d4. Latest consensus state: current_term: 2 leader_uuid: "1a26024e37064a939ea8231ab26595d4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: false } } }
I20260811 22:19:12.132777 11500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:19:12.135067 11514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "1a26024e37064a939ea8231ab26595d4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: false } } }
I20260811 22:19:12.135434 11514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:12.135775 11500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "1a26024e37064a939ea8231ab26595d4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "fc323e8e063549859af13dc2f68c4f2f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43683 } } peers { permanent_uuid: "6ae333e2d30e4a2e90c526cfbdfc1b79" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45437 } attrs { promote: false } } peers { permanent_uuid: "1a26024e37064a939ea8231ab26595d4" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 37361 } attrs { promote: false } } }
I20260811 22:19:12.136183 11500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae333e2d30e4a2e90c526cfbdfc1b79 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:19:12.156770 11502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.145645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:45275 (local address 127.0.0.1:34270)
0811 22:19:12.145849 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.145865 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:12.145996 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:12.146934 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:12.146940 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:12.146968 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:12.147437 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:12.147455 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.149602 (+ 2147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.149610 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:12.152450 (+ 2840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:12.152471 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.155611 (+ 3140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.155621 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:12.155933 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:12.155994 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 22:19:12.156086 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 22:19:12.156313 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 22:19:12.158833 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.145829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34270 (local address 127.29.229.1:45275)
0811 22:19:12.146419 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:12.146426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:12.146445 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:19:12.146518 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:12.146522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:12.146608 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:12.147633 (+ 1025us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:12.147879 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.149390 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.152753 (+ 3363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.155213 (+ 2460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.157907 (+ 2694us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:12.158061 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:12.158131 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 22:19:12.158358 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":315,"threads_started":1}
I20260811 22:19:12.159271 11503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.145250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:41123 (local address 127.0.0.1:40034)
0811 22:19:12.145435 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.145457 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:12.145582 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:12.148210 (+ 2628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:12.148218 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:12.148257 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:12.148542 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:12.148560 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.152018 (+ 3458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.152027 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:12.154627 (+ 2600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:12.154648 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.158337 (+ 3689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.158347 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:12.158679 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:12.158731 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 22:19:12.158807 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 22:19:12.158968 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 22:19:12.159937 10817 tablet_service.cc:1467] Processing CreateTablet for tablet 9b4ee03647534b9f83caa4d90b7660d7 (DEFAULT_TABLE table=test-table [id=d41b9862dc8248ff9cb3de74baf079ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:12.160516 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b4ee03647534b9f83caa4d90b7660d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:12.160265 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.145435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40034 (local address 127.29.229.2:41123)
0811 22:19:12.146587 (+ 1152us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:12.146595 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:12.146618 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:19:12.146695 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:12.146699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:12.148003 (+ 1304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:12.148723 (+ 720us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:12.148946 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.150381 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.154922 (+ 4541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.158108 (+ 3186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.159614 (+ 1506us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:12.159732 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:12.159786 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 22:19:12.159939 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1016,"thread_start_us":908,"threads_started":1}
I20260811 22:19:12.162147 11042 tablet_service.cc:1467] Processing CreateTablet for tablet 9b4ee03647534b9f83caa4d90b7660d7 (DEFAULT_TABLE table=test-table [id=d41b9862dc8248ff9cb3de74baf079ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:12.162623 11042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b4ee03647534b9f83caa4d90b7660d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:12.165798 11502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.146792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:46419 (local address 127.0.0.1:52198)
0811 22:19:12.157098 (+ 10306us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.157123 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:12.157251 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:12.159884 (+ 2633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:12.159889 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:12.159916 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:12.160084 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:12.160097 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.161587 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.161593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:12.163359 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:12.163376 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.164988 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.164994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:12.165203 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:12.165238 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 22:19:12.165348 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 22:19:12.165489 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10169,"thread_start_us":12206,"threads_started":1}
I20260811 22:19:12.165843 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.146943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52198 (local address 127.29.229.3:46419)
0811 22:19:12.159560 (+ 12617us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:12.159567 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:12.159587 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:19:12.159661 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:12.159668 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:12.159749 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:12.159858 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:12.160257 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.161415 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.163577 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.164808 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.164939 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:12.165056 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:12.165399 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 22:19:12.165548 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12466,"thread_start_us":12359,"threads_started":1}
I20260811 22:19:12.166774 11531 tablet_bootstrap.cc:492] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31: Bootstrap starting.
I20260811 22:19:12.166955 11274 tablet_service.cc:1467] Processing CreateTablet for tablet 9b4ee03647534b9f83caa4d90b7660d7 (DEFAULT_TABLE table=test-table [id=d41b9862dc8248ff9cb3de74baf079ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:12.167414 11274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b4ee03647534b9f83caa4d90b7660d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:12.168840 11531 tablet_bootstrap.cc:654] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:12.171475 11536 tablet_bootstrap.cc:492] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee: Bootstrap starting.
I20260811 22:19:12.173453 11536 tablet_bootstrap.cc:654] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:12.174253 11531 tablet_bootstrap.cc:492] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31: No bootstrap required, opened a new log
I20260811 22:19:12.174453 11531 ts_tablet_manager.cc:1397] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 22:19:12.175480 11531 raft_consensus.cc:348] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } } peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } }
I20260811 22:19:12.175822 11536 tablet_bootstrap.cc:492] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee: No bootstrap required, opened a new log
I20260811 22:19:12.175905 11531 raft_consensus.cc:374] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:12.175979 11536 ts_tablet_manager.cc:1397] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 22:19:12.176050 11531 raft_consensus.cc:729] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be34d3c2b4c5470caf70aac065cc5d31, State: Initialized, Role: FOLLOWER
I20260811 22:19:12.176371 11531 consensus_queue.cc:260] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } } peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } }
I20260811 22:19:12.176925 11536 raft_consensus.cc:348] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } } peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } }
I20260811 22:19:12.177174 11536 raft_consensus.cc:374] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:12.177213 11539 tablet_bootstrap.cc:492] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf: Bootstrap starting.
I20260811 22:19:12.177284 11536 raft_consensus.cc:729] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 655348f7beb645b98a09af862a80c1ee, State: Initialized, Role: FOLLOWER
I20260811 22:19:12.177532 11536 consensus_queue.cc:260] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } } peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } }
I20260811 22:19:12.178450 11536 ts_tablet_manager.cc:1428] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 22:19:12.179440 11539 tablet_bootstrap.cc:654] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:12.185488 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.180225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.2:34625)
0811 22:19:12.184894 (+ 4669us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.185111 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4432,"thread_start_us":4258,"threads_started":1}
I20260811 22:19:12.186601 11531 ts_tablet_manager.cc:1428] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31: Time spent starting tablet: real 0.012s user 0.008s sys 0.000s
I20260811 22:19:12.186681 11148 heartbeater.cc:503] Master 127.29.229.60:37361 was elected leader, sending a full tablet report...
I20260811 22:19:12.187443 11539 tablet_bootstrap.cc:492] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf: No bootstrap required, opened a new log
I20260811 22:19:12.187701 11539 ts_tablet_manager.cc:1397] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260811 22:19:12.188311 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.183884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.1:38233)
0811 22:19:12.187358 (+ 3474us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.188012 (+ 654us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3315,"thread_start_us":3079,"threads_started":1}
I20260811 22:19:12.188910 11539 raft_consensus.cc:348] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } } peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } }
I20260811 22:19:12.189703 11539 raft_consensus.cc:374] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:12.189949 11539 raft_consensus.cc:729] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43ca70cae5964345831bcafd0d7178bf, State: Initialized, Role: FOLLOWER
I20260811 22:19:12.190441 10928 heartbeater.cc:503] Master 127.29.229.60:37361 was elected leader, sending a full tablet report...
I20260811 22:19:12.190629 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.190222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.1:41751)
0811 22:19:12.190371 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.190477 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 22:19:12.190409 11539 consensus_queue.cc:260] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } } peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } }
I20260811 22:19:12.191589 11539 ts_tablet_manager.cc:1428] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 22:19:12.194582 11550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.192601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.3:38339)
0811 22:19:12.193501 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.193683 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":565,"threads_started":1}
I20260811 22:19:12.194983 11376 heartbeater.cc:503] Master 127.29.229.60:37361 was elected leader, sending a full tablet report...
I20260811 22:19:12.195350 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.194620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.2:56051)
0811 22:19:12.194797 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.195178 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 22:19:12.196236 11550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.195722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.3:36551)
0811 22:19:12.195994 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.196076 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260811 22:19:12.239323 11541 raft_consensus.cc:482] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:12.239632 11541 raft_consensus.cc:504] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } } peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } }
I20260811 22:19:12.240540 11541 leader_election.cc:288] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 655348f7beb645b98a09af862a80c1ee (127.29.229.2:41123), 43ca70cae5964345831bcafd0d7178bf (127.29.229.3:46419)
I20260811 22:19:12.241221 11074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b4ee03647534b9f83caa4d90b7660d7" candidate_uuid: "be34d3c2b4c5470caf70aac065cc5d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "655348f7beb645b98a09af862a80c1ee" is_pre_election: true
I20260811 22:19:12.241488 11074 raft_consensus.cc:2454] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be34d3c2b4c5470caf70aac065cc5d31 in term 0.
I20260811 22:19:12.241544 11305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b4ee03647534b9f83caa4d90b7660d7" candidate_uuid: "be34d3c2b4c5470caf70aac065cc5d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43ca70cae5964345831bcafd0d7178bf" is_pre_election: true
I20260811 22:19:12.241876 11305 raft_consensus.cc:2454] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be34d3c2b4c5470caf70aac065cc5d31 in term 0.
I20260811 22:19:12.242003 10742 leader_election.cc:302] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 655348f7beb645b98a09af862a80c1ee, be34d3c2b4c5470caf70aac065cc5d31; no voters:
I20260811 22:19:12.242332 11541 raft_consensus.cc:2781] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:19:12.242480 11541 raft_consensus.cc:482] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:19:12.242607 11541 raft_consensus.cc:3037] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:12.244827 11541 raft_consensus.cc:504] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } } peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } }
I20260811 22:19:12.245457 11541 leader_election.cc:288] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [CANDIDATE]: Term 1 election: Requested vote from peers 655348f7beb645b98a09af862a80c1ee (127.29.229.2:41123), 43ca70cae5964345831bcafd0d7178bf (127.29.229.3:46419)
I20260811 22:19:12.246011 11305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b4ee03647534b9f83caa4d90b7660d7" candidate_uuid: "be34d3c2b4c5470caf70aac065cc5d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43ca70cae5964345831bcafd0d7178bf"
I20260811 22:19:12.246004 11074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b4ee03647534b9f83caa4d90b7660d7" candidate_uuid: "be34d3c2b4c5470caf70aac065cc5d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "655348f7beb645b98a09af862a80c1ee"
I20260811 22:19:12.246248 11074 raft_consensus.cc:3037] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:12.246249 11305 raft_consensus.cc:3037] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:12.249418 11074 raft_consensus.cc:2454] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be34d3c2b4c5470caf70aac065cc5d31 in term 1.
I20260811 22:19:12.249449 11305 raft_consensus.cc:2454] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be34d3c2b4c5470caf70aac065cc5d31 in term 1.
I20260811 22:19:12.249922 10741 leader_election.cc:302] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43ca70cae5964345831bcafd0d7178bf, be34d3c2b4c5470caf70aac065cc5d31; no voters:
I20260811 22:19:12.250276 11541 raft_consensus.cc:2781] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:19:12.250788 11541 raft_consensus.cc:686] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [term 1 LEADER]: Becoming Leader. State: Replica: be34d3c2b4c5470caf70aac065cc5d31, State: Running, Role: LEADER
I20260811 22:19:12.251160 11541 consensus_queue.cc:237] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } } peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } }
I20260811 22:19:12.252426 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.251566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.1:34895)
0811 22:19:12.251953 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.252151 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260811 22:19:12.254828 10621 catalog_manager.cc:5495] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 reported cstate change: term changed from 0 to 1, leader changed from <none> to be34d3c2b4c5470caf70aac065cc5d31 (127.29.229.1). New cstate: current_term: 1 leader_uuid: "be34d3c2b4c5470caf70aac065cc5d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be34d3c2b4c5470caf70aac065cc5d31" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 } health_report { overall_health: UNKNOWN } } }
I20260811 22:19:12.289386 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.283512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34278 (local address 127.29.229.1:45275)
0811 22:19:12.283728 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:12.283733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:12.283751 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:19:12.283818 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:12.283830 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:12.283895 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:12.283973 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:19:12.284397 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.285255 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.287170 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.287648 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.287758 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:12.288345 (+ 587us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:19:12.288616 (+ 271us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:19:12.288738 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:12.288903 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 22:19:12.289013 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:19:12.293403 11305 raft_consensus.cc:1261] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf [term 1 FOLLOWER]: Refusing update from remote peer be34d3c2b4c5470caf70aac065cc5d31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:12.294205 11074 raft_consensus.cc:1261] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee [term 1 FOLLOWER]: Refusing update from remote peer be34d3c2b4c5470caf70aac065cc5d31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:12.294179 11552 consensus_queue.cc:1035] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43ca70cae5964345831bcafd0d7178bf" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 46419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:12.294806 11552 consensus_queue.cc:1035] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "655348f7beb645b98a09af862a80c1ee" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 41123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:12.297169 11550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.296350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.3:41345)
0811 22:19:12.296833 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.296962 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361}
I20260811 22:19:12.298745 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.298255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.2:57535)
0811 22:19:12.298417 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.298547 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 22:19:12.306196 11556 mvcc.cc:205] Tried to move back new op lower bound from 7317449737386872832 to 7317449737223319552. Current Snapshot: MvccSnapshot[applied={T|T < 7317449737386872832}]
I20260811 22:19:12.308084 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.307419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.2:47555)
0811 22:19:12.307777 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.307892 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238}
I20260811 22:19:12.309372 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.308845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.1:56907)
0811 22:19:12.309087 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.309196 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260811 22:19:12.315589 10620 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:12.315943 10620 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:12.316294 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.315667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.1:43843)
0811 22:19:12.315991 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.316103 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202}
I20260811 22:19:12.319546 11550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.318775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.29.229.3:37325)
0811 22:19:12.319225 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.319328 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336}
I20260811 22:19:12.324282 10620 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a26024e37064a939ea8231ab26595d4: Sending DeleteTablet for 3 replicas of tablet 9b4ee03647534b9f83caa4d90b7660d7
I20260811 22:19:12.325739 11042 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b4ee03647534b9f83caa4d90b7660d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:12 UTC) from {username='oryx', principal='oryx/127.29.229.60@KRBTEST.COM'} at 127.0.0.1:40034
I20260811 22:19:12.325737 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b4ee03647534b9f83caa4d90b7660d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:12 UTC) from {username='oryx', principal='oryx/127.29.229.60@KRBTEST.COM'} at 127.0.0.1:34270
I20260811 22:19:12.326169 10815 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b4ee03647534b9f83caa4d90b7660d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:12 UTC) from {username='oryx', principal='oryx/127.29.229.60@KRBTEST.COM'} at 127.0.0.1:34270
I20260811 22:19:12.326973 11565 tablet_replica.cc:330] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31: stopping tablet replica
I20260811 22:19:12.327190 11564 tablet_replica.cc:330] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee: stopping tablet replica
I20260811 22:19:12.327215 11565 raft_consensus.cc:2229] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:19:12.327389 11564 raft_consensus.cc:2229] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:12.327644 11565 raft_consensus.cc:2258] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:12.327628 11274 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b4ee03647534b9f83caa4d90b7660d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:12 UTC) from {username='oryx', principal='oryx/127.29.229.60@KRBTEST.COM'} at 127.0.0.1:52198
I20260811 22:19:12.327730 11564 raft_consensus.cc:2258] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:12.327965 11273 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b4ee03647534b9f83caa4d90b7660d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:12 UTC) from {username='oryx', principal='oryx/127.29.229.60@KRBTEST.COM'} at 127.0.0.1:52198
I20260811 22:19:12.328963 11567 tablet_replica.cc:330] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf: stopping tablet replica
I20260811 22:19:12.329981 11565 ts_tablet_manager.cc:1905] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:12.330415 11564 ts_tablet_manager.cc:1905] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:12.330464 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 10595
I20260811 22:19:12.334137 11567 raft_consensus.cc:2229] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:12.334537 11567 raft_consensus.cc:2258] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:12.335749 11567 ts_tablet_manager.cc:1905] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:12.339548 11565 ts_tablet_manager.cc:1918] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:12.339720 11565 log.cc:1196] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-0/wal/wals/9b4ee03647534b9f83caa4d90b7660d7
I20260811 22:19:12.340224 11565 ts_tablet_manager.cc:1939] T 9b4ee03647534b9f83caa4d90b7660d7 P be34d3c2b4c5470caf70aac065cc5d31: Deleting consensus metadata
I20260811 22:19:12.340369 11564 ts_tablet_manager.cc:1918] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:12.340585 11564 log.cc:1196] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/9b4ee03647534b9f83caa4d90b7660d7
I20260811 22:19:12.341151 11564 ts_tablet_manager.cc:1939] T 9b4ee03647534b9f83caa4d90b7660d7 P 655348f7beb645b98a09af862a80c1ee: Deleting consensus metadata
I20260811 22:19:12.341708 10600 catalog_manager.cc:4841] TS be34d3c2b4c5470caf70aac065cc5d31 (127.29.229.1:45275): tablet 9b4ee03647534b9f83caa4d90b7660d7 (table test-table [id=d41b9862dc8248ff9cb3de74baf079ff]) successfully deleted
I20260811 22:19:12.342646 11567 ts_tablet_manager.cc:1918] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:12.342700 10603 catalog_manager.cc:4841] TS 655348f7beb645b98a09af862a80c1ee (127.29.229.2:41123): tablet 9b4ee03647534b9f83caa4d90b7660d7 (table test-table [id=d41b9862dc8248ff9cb3de74baf079ff]) successfully deleted
I20260811 22:19:12.342806 11567 log.cc:1196] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/9b4ee03647534b9f83caa4d90b7660d7
W20260811 22:19:12.342912 10600 catalog_manager.cc:4808] TS be34d3c2b4c5470caf70aac065cc5d31 (127.29.229.1:45275): delete failed for tablet 9b4ee03647534b9f83caa4d90b7660d7 because the tablet was not found. No further retry: Not found: Tablet not found: 9b4ee03647534b9f83caa4d90b7660d7
I20260811 22:19:12.343343 11567 ts_tablet_manager.cc:1939] T 9b4ee03647534b9f83caa4d90b7660d7 P 43ca70cae5964345831bcafd0d7178bf: Deleting consensus metadata
I20260811 22:19:12.344579 10602 catalog_manager.cc:4841] TS 43ca70cae5964345831bcafd0d7178bf (127.29.229.3:46419): tablet 9b4ee03647534b9f83caa4d90b7660d7 (table test-table [id=d41b9862dc8248ff9cb3de74baf079ff]) successfully deleted
W20260811 22:19:12.344871 10602 catalog_manager.cc:4808] TS 43ca70cae5964345831bcafd0d7178bf (127.29.229.3:46419): delete failed for tablet 9b4ee03647534b9f83caa4d90b7660d7 because the tablet was not found. No further retry: Not found: Tablet not found: 9b4ee03647534b9f83caa4d90b7660d7
I20260811 22:19:12.360711 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.331590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34288 (local address 127.29.229.1:45275)
0811 22:19:12.331799 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:12.331809 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:12.343734 (+ 11925us) server_negotiation.cc:407] Connection header received
0811 22:19:12.343930 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:12.343934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:12.344025 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:12.344118 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:12.345782 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.346531 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.347785 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.348235 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.348376 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:12.352166 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:12.352207 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:12.352242 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:12.352432 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:12.357394 (+ 4962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:12.358340 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:12.358350 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:12.358353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:12.358549 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:12.358963 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:12.358968 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:12.358972 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:12.359447 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:12.359777 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:12.360060 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 22:19:12.360303 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:19:12.361596 10920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:12.490280 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 10595
I20260811 22:19:12.527032 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 10946
I20260811 22:19:12.529439 11502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.529002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.0.0.1:58140)
0811 22:19:12.529173 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.529257 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 22:19:12.530916 11502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.530167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.0.0.1:58152)
0811 22:19:12.530533 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.530739 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260811 22:19:12.543993 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.527913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40050 (local address 127.29.229.2:41123)
0811 22:19:12.528231 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:12.528237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:12.528262 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:19:12.528462 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:12.528468 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:12.528594 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:12.528719 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:12.530703 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.531901 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.533548 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.534014 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.534140 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:12.537805 (+ 3665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:12.537822 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:12.537839 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:12.537878 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:12.540751 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:12.541734 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:12.541741 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:12.541744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:12.541874 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:12.542288 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:12.542293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:12.542296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:12.542738 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:12.542971 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:12.543400 (+ 429us) server_negotiation.cc:299] Negotiation successful
0811 22:19:12.543607 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260811 22:19:12.544781 11138 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:12.644336 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 10946
I20260811 22:19:12.672115 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 11170
I20260811 22:19:12.688781 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.673018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52208 (local address 127.29.229.3:46419)
0811 22:19:12.673575 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:12.673579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:12.673596 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:19:12.673653 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:12.673656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:12.673717 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:12.673811 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:12.675343 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.676216 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.678046 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:12.678475 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:12.678592 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:12.681965 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:12.682006 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:12.682034 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:12.682109 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:12.685448 (+ 3339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:12.686349 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:12.686358 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:12.686362 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:12.686510 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:12.686966 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:12.686970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:12.686972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:12.687414 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:12.687675 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:12.688214 (+ 539us) server_negotiation.cc:299] Negotiation successful
0811 22:19:12.688412 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":297,"threads_started":1}
I20260811 22:19:12.689697 11371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:12.787506 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 11170
I20260811 22:19:12.814999 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 10093
I20260811 22:19:12.816152 10357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:12.937922 11502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.937471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.0.0.1:58162)
0811 22:19:12.937657 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.937753 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 22:19:12.938956 11502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:12.938596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43683 (local address 127.0.0.1:58176)
0811 22:19:12.938751 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:12.938849 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:43683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 22:19:12.941056 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 10093
I20260811 22:19:12.970314 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 10253
I20260811 22:19:12.971442 10670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:13.107849 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 10253
2026-08-11T22:19:13Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10996 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:19:13 dist-test-slave-jq4t krb5kdc[11582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:19:13 dist-test-slave-jq4t krb5kdc[11582](info): set up 2 sockets
Aug 11 22:19:13 dist-test-slave-jq4t krb5kdc[11582](info): commencing operation
krb5kdc: starting...
W20260811 22:19:14.543696 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.368s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:19:14 dist-test-slave-jq4t krb5kdc[11582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:19:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:19:14Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.62.keytab.
Entry for principal oryx/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.62.keytab.
I20260811 22:19:14.861281 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:43617
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.62.keytab
--principal=oryx/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:38857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:43617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:15.254616 11604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:15.254920 11604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:15.255069 11604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:15.262704 11604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:19:15.262816 11604 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:19:15.262864 11604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:15.262979 11604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:15.263074 11604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:19:15.263182 11604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:19:15.273404 11604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:38857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:43617
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.62.keytab
--principal=oryx/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:43617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:15.274767 11604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:15.276180 11604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:15.288707 11611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:15.288582 11616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:15.288655 11612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:15.289731 11604 server_base.cc:1034] running on GCE node
I20260811 22:19:15.290555 11604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:15.292316 11604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:15.293603 11604 hybrid_clock.cc:648] HybridClock initialized: now 1786486755293523 us; error 58 us; skew 500 ppm
I20260811 22:19:15.294006 11604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:15 dist-test-slave-jq4t krb5kdc[11582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486755, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:15.297506 11604 init.cc:377] Logged in from keytab as oryx/127.29.229.62@KRBTEST.COM (short username oryx)
I20260811 22:19:15.299185 11604 webserver.cc:460] Webserver started at http://127.29.229.62:40783/ using document root <none> and password file <none>
I20260811 22:19:15.299965 11604 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:15.300119 11604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:15.300464 11604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:15.304111 11604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "04f4c66999bd4c0cb9e040becfb76847"
format_stamp: "Formatted at 2026-08-11 22:19:15 on dist-test-slave-jq4t"
server_key: "27b4897d213c1640b586c312b3c3583f"
server_key_iv: "d09b3c152ff342708d1a8838bfbd984d"
server_key_version: "encryptionkey@0"
I20260811 22:19:15.305004 11604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "04f4c66999bd4c0cb9e040becfb76847"
format_stamp: "Formatted at 2026-08-11 22:19:15 on dist-test-slave-jq4t"
server_key: "27b4897d213c1640b586c312b3c3583f"
server_key_iv: "d09b3c152ff342708d1a8838bfbd984d"
server_key_version: "encryptionkey@0"
I20260811 22:19:15.311082 11604 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260811 22:19:15.315027 11628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:15.316382 11604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:19:15.316609 11604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "04f4c66999bd4c0cb9e040becfb76847"
format_stamp: "Formatted at 2026-08-11 22:19:15 on dist-test-slave-jq4t"
server_key: "27b4897d213c1640b586c312b3c3583f"
server_key_iv: "d09b3c152ff342708d1a8838bfbd984d"
server_key_version: "encryptionkey@0"
I20260811 22:19:15.316861 11604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:15.332721 11604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:15.337296 11604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:15.337595 11604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:15.361297 11604 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:43617
I20260811 22:19:15.361335 11716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:43617 every 8 connection(s)
I20260811 22:19:15.362810 11604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:19:15.365744 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 11604
I20260811 22:19:15.366003 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:19:15.366863 30612 external_mini_cluster.cc:1428] Setting key 0d9ea3570b163c6a9face93899e97215
I20260811 22:19:15.367427 11717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:15.383481 11717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847: Bootstrap starting.
Aug 11 22:19:15 dist-test-slave-jq4t krb5kdc[11582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM
I20260811 22:19:15.388309 11717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:15.389868 11717 log.cc:824] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:15.393702 11717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847: No bootstrap required, opened a new log
I20260811 22:19:15.394546 11720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:15.369141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50762 (local address 127.29.229.62:43617)
0811 22:19:15.369971 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:15.369982 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:15.370023 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:19:15.372388 (+ 2365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:15.372412 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:15.372890 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:15.373225 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:15.375158 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:15.376702 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:15.378579 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:15.379580 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:15.379826 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:15.385635 (+ 5809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:15.385648 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:15.385671 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:15.385728 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:15.389663 (+ 3935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:15.390556 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:15.390565 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:15.390574 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:15.390779 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:15.391209 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:15.391214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:15.391218 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:15.391801 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:15.392116 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:15.392825 (+ 709us) server_negotiation.cc:299] Negotiation successful
0811 22:19:15.393388 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":402,"threads_started":1}
I20260811 22:19:15.402535 11717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f4c66999bd4c0cb9e040becfb76847" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43617 } }
I20260811 22:19:15.402999 11717 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:15.403119 11717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04f4c66999bd4c0cb9e040becfb76847, State: Initialized, Role: FOLLOWER
I20260811 22:19:15.403729 11717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f4c66999bd4c0cb9e040becfb76847" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43617 } }
I20260811 22:19:15.403999 11717 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:19:15.404155 11717 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:19:15.404333 11717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:15.406730 11717 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f4c66999bd4c0cb9e040becfb76847" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43617 } }
I20260811 22:19:15.407351 11717 leader_election.cc:302] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04f4c66999bd4c0cb9e040becfb76847; no voters:
I20260811 22:19:15.408200 11717 leader_election.cc:288] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:19:15.408517 11723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:19:15.409385 11723 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847 [term 1 LEADER]: Becoming Leader. State: Replica: 04f4c66999bd4c0cb9e040becfb76847, State: Running, Role: LEADER
I20260811 22:19:15.410096 11723 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f4c66999bd4c0cb9e040becfb76847" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43617 } }
I20260811 22:19:15.410919 11717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:19:15.413834 11726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04f4c66999bd4c0cb9e040becfb76847" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f4c66999bd4c0cb9e040becfb76847" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43617 } } }
I20260811 22:19:15.414268 11726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:15.414258 11728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04f4c66999bd4c0cb9e040becfb76847. Latest consensus state: current_term: 1 leader_uuid: "04f4c66999bd4c0cb9e040becfb76847" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f4c66999bd4c0cb9e040becfb76847" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43617 } } }
I20260811 22:19:15.414647 11728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:15.416848 11740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:19:15.422323 11740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:19:15.432056 11740 catalog_manager.cc:1347] Generated new cluster ID: 82015e918089450bb051817e619fe79e
I20260811 22:19:15.432224 11740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:19:15.443900 11740 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:19:15.445876 11740 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:19:15.455436 11740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847: Generated new TSK 0
I20260811 22:19:15.456548 11740 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.1.keytab.
Entry for principal oryx/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.1.keytab.
I20260811 22:19:15.535928 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43617
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.1.keytab
--principal=oryx/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:38857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:15.906780 11765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:15.907116 11765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:15.907312 11765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:15.915503 11765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:15.915665 11765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:15.915844 11765 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:19:15.915992 11765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:19:15.927872 11765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:38857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.1.keytab
--principal=oryx/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.229.62:43617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:15.929400 11765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:15.931070 11765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:15.946189 11773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:15.946132 11780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:15.946215 11776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:15.947036 11765 server_base.cc:1034] running on GCE node
I20260811 22:19:15.947722 11765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:15.948928 11765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:15.950237 11765 hybrid_clock.cc:648] HybridClock initialized: now 1786486755950157 us; error 47 us; skew 500 ppm
I20260811 22:19:15.950591 11765 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:15 dist-test-slave-jq4t krb5kdc[11582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486755, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:15.954290 11765 init.cc:377] Logged in from keytab as oryx/127.29.229.1@KRBTEST.COM (short username oryx)
I20260811 22:19:15.955971 11765 webserver.cc:460] Webserver started at http://127.29.229.1:46577/ using document root <none> and password file <none>
I20260811 22:19:15.956809 11765 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:15.956910 11765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:15.957249 11765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:15.960377 11765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "99db8de4f9d94255bde71c19700597ae"
format_stamp: "Formatted at 2026-08-11 22:19:15 on dist-test-slave-jq4t"
server_key: "515ffba312dfdbab74fcaf35d7fe6fa3"
server_key_iv: "4e67e3adb3a600e64e349977f1bbfbdf"
server_key_version: "encryptionkey@0"
I20260811 22:19:15.961256 11765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "99db8de4f9d94255bde71c19700597ae"
format_stamp: "Formatted at 2026-08-11 22:19:15 on dist-test-slave-jq4t"
server_key: "515ffba312dfdbab74fcaf35d7fe6fa3"
server_key_iv: "4e67e3adb3a600e64e349977f1bbfbdf"
server_key_version: "encryptionkey@0"
I20260811 22:19:15.967722 11765 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 22:19:15.972633 11793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:15.974400 11765 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:19:15.974588 11765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "99db8de4f9d94255bde71c19700597ae"
format_stamp: "Formatted at 2026-08-11 22:19:15 on dist-test-slave-jq4t"
server_key: "515ffba312dfdbab74fcaf35d7fe6fa3"
server_key_iv: "4e67e3adb3a600e64e349977f1bbfbdf"
server_key_version: "encryptionkey@0"
I20260811 22:19:15.974826 11765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:15.994539 11765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:16.000552 11765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:16.000830 11765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:16.003837 11765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:16.003978 11765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:16.004207 11765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:16.004319 11765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:19:16 dist-test-slave-jq4t krb5kdc[11582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486755, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM
I20260811 22:19:16.032437 11811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:16.004386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:49663 (local address 127.29.229.62:43617)
0811 22:19:16.005099 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:16.005108 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:16.006225 (+ 1117us) server_negotiation.cc:407] Connection header received
0811 22:19:16.009282 (+ 3057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:16.009289 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:16.009450 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:16.009601 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:16.013087 (+ 3486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:16.014163 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:16.018441 (+ 4278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:16.019025 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:16.019208 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:16.024936 (+ 5728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:16.024942 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:16.024945 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:16.024984 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:16.028622 (+ 3638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:16.029810 (+ 1188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:16.029816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:16.029820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:16.029966 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:16.030465 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:16.030470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:16.030474 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:16.030781 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:16.030966 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:16.031758 (+ 792us) server_negotiation.cc:299] Negotiation successful
0811 22:19:16.032011 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":444,"threads_started":1}
I20260811 22:19:16.033730 11818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:16.005171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43617 (local address 127.29.229.1:49663)
0811 22:19:16.006046 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:16.006087 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:16.008749 (+ 2662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:16.009983 (+ 1234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:16.009994 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:16.011510 (+ 1516us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:16.012741 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:16.012785 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:16.014455 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:16.014469 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:16.018149 (+ 3680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:16.018180 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:16.019373 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:16.019386 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:16.019781 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:16.020788 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:16.020825 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:16.024615 (+ 3790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:16.028881 (+ 4266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:16.028892 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:16.028911 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:16.029651 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:16.030089 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:16.030094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:16.030098 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:16.030333 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:16.030935 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:16.030944 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:16.031362 (+ 418us) client_negotiation.cc:769] Sending connection context
0811 22:19:16.031643 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 22:19:16.032246 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":374,"threads_started":1}
I20260811 22:19:16.061522 11811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:16.041277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:51241 (local address 127.29.229.62:43617)
0811 22:19:16.041485 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:16.041491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:16.042557 (+ 1066us) server_negotiation.cc:407] Connection header received
0811 22:19:16.042688 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:16.042696 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:16.042822 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:16.042982 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:16.043954 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:16.045303 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:16.048302 (+ 2999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:16.049158 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:16.049331 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:16.053091 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:16.053098 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:16.053103 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:16.053161 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:16.057713 (+ 4552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:16.059063 (+ 1350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:16.059073 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:16.059080 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:16.059279 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:16.059833 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:16.059840 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:16.059843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:16.060198 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:16.060393 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:16.060885 (+ 492us) server_negotiation.cc:299] Negotiation successful
0811 22:19:16.061143 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:19:16.061822 11929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:16.041489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43617 (local address 127.29.229.1:51241)
0811 22:19:16.042433 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:16.042450 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:16.042575 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:16.043020 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:16.043023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:16.043501 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:16.043801 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:16.043810 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:16.045527 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:16.045536 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:16.048031 (+ 2495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:16.048060 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:16.049698 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:16.049710 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:16.049970 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:16.051049 (+ 1079us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:16.051080 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:16.052663 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:16.057949 (+ 5286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:16.057959 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:16.057964 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:16.058837 (+ 873us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:16.059455 (+ 618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:16.059462 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:16.059468 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:16.059680 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:16.060341 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:16.060348 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:16.060614 (+ 266us) client_negotiation.cc:769] Sending connection context
0811 22:19:16.060775 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 22:19:16.061285 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"thread_start_us":487,"threads_started":1}
I20260811 22:19:16.064634 11765 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:38279
I20260811 22:19:16.064733 12004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:38279 every 8 connection(s)
I20260811 22:19:16.066902 11765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:19:16.069339 12005 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43617
I20260811 22:19:16.069608 12005 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:16.070505 12005 heartbeater.cc:511] Master 127.29.229.62:43617 requested a full tablet report, sending...
I20260811 22:19:16.070643 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 11765
I20260811 22:19:16.070881 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:19:16.071722 30612 external_mini_cluster.cc:1428] Setting key 7b75d18938f5f1815ed6851ffdd44589
I20260811 22:19:16.073156 11655 ts_manager.cc:194] Registered new tserver with Master: 99db8de4f9d94255bde71c19700597ae (127.29.229.1:38279)
I20260811 22:19:16.074759 11655 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:49663
WARNING: no policy specified for oryx/127.29.229.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.229.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.2.keytab.
Entry for principal oryx/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.2.keytab.
I20260811 22:19:16.143792 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43617
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.2.keytab
--principal=oryx/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:38857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:16.519958 12012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:16.520385 12012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:16.520586 12012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:16.528283 12012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:16.528445 12012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:16.528616 12012 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:19:16.528756 12012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:19:16.539273 12012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:38857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.2.keytab
--principal=oryx/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.229.62:43617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:16.540722 12012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:16.542140 12012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:16.556182 12018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:16.556178 12022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:16.556178 12019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:16.557127 12012 server_base.cc:1034] running on GCE node
I20260811 22:19:16.557772 12012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:16.558758 12012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:16.560039 12012 hybrid_clock.cc:648] HybridClock initialized: now 1786486756559968 us; error 40 us; skew 500 ppm
I20260811 22:19:16.560410 12012 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:16 dist-test-slave-jq4t krb5kdc[11582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486756, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:16.564919 12012 init.cc:377] Logged in from keytab as oryx/127.29.229.2@KRBTEST.COM (short username oryx)
I20260811 22:19:16.566476 12012 webserver.cc:460] Webserver started at http://127.29.229.2:40045/ using document root <none> and password file <none>
I20260811 22:19:16.567229 12012 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:16.567366 12012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:16.567724 12012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:16.570785 12012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "6418cf43a4284bf397afb7296ab76b2b"
format_stamp: "Formatted at 2026-08-11 22:19:16 on dist-test-slave-jq4t"
server_key: "61083990b4d9d398792cde05f40bb461"
server_key_iv: "c0d259b4a5d61434e53120fd3cc5ac7b"
server_key_version: "encryptionkey@0"
I20260811 22:19:16.571645 12012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "6418cf43a4284bf397afb7296ab76b2b"
format_stamp: "Formatted at 2026-08-11 22:19:16 on dist-test-slave-jq4t"
server_key: "61083990b4d9d398792cde05f40bb461"
server_key_iv: "c0d259b4a5d61434e53120fd3cc5ac7b"
server_key_version: "encryptionkey@0"
I20260811 22:19:16.577230 12012 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260811 22:19:16.581084 12032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:16.582469 12012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 22:19:16.582649 12012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "6418cf43a4284bf397afb7296ab76b2b"
format_stamp: "Formatted at 2026-08-11 22:19:16 on dist-test-slave-jq4t"
server_key: "61083990b4d9d398792cde05f40bb461"
server_key_iv: "c0d259b4a5d61434e53120fd3cc5ac7b"
server_key_version: "encryptionkey@0"
I20260811 22:19:16.582897 12012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:16.646440 12012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:16.651589 12012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:16.651916 12012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:16.654950 12012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:16.655092 12012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:16.655254 12012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:16.655346 12012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:19:16 dist-test-slave-jq4t krb5kdc[11582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486756, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM
I20260811 22:19:16.683270 12046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:16.655834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:53135 (local address 127.29.229.62:43617)
0811 22:19:16.656342 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:16.656347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:16.658351 (+ 2004us) server_negotiation.cc:407] Connection header received
0811 22:19:16.661912 (+ 3561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:16.661919 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:16.662027 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:16.662170 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:16.664733 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:16.665894 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:16.668787 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:16.669701 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:16.669870 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:16.675380 (+ 5510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:16.675385 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:16.675388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:16.675427 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:16.679182 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:16.680134 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:16.680139 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:16.680143 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:16.680311 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:16.680932 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:16.680937 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:16.680940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:16.681248 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:16.681450 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:16.682550 (+ 1100us) server_negotiation.cc:299] Negotiation successful
0811 22:19:16.682839 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":288,"threads_started":1}
I20260811 22:19:16.687326 12047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:16.656704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43617 (local address 127.29.229.2:53135)
0811 22:19:16.658020 (+ 1316us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:16.658108 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:16.661517 (+ 3409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:16.662482 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:16.662491 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:16.663494 (+ 1003us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:16.664470 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:16.664559 (+ 89us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:16.666085 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:16.666089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:16.668462 (+ 2373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:16.668525 (+ 63us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:16.670011 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:16.670020 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:16.670335 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:16.671248 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:16.671275 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:16.675036 (+ 3761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:16.679370 (+ 4334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:16.679385 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:16.679404 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:16.679995 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:16.680417 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:16.680423 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:16.680427 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:16.680786 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:16.681420 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:16.681429 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:16.681982 (+ 553us) client_negotiation.cc:769] Sending connection context
0811 22:19:16.683614 (+ 1632us) client_negotiation.cc:240] Negotiation successful
0811 22:19:16.684323 (+ 709us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"thread_start_us":561,"threads_started":1}
I20260811 22:19:16.704583 12012 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:36507
I20260811 22:19:16.705219 12218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:36507 every 8 connection(s)
I20260811 22:19:16.706766 12012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:19:16.709975 12219 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43617
I20260811 22:19:16.710217 12219 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:16.710937 12219 heartbeater.cc:511] Master 127.29.229.62:43617 requested a full tablet report, sending...
I20260811 22:19:16.711763 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 12012
I20260811 22:19:16.711966 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:19:16.712648 30612 external_mini_cluster.cc:1428] Setting key 4b2213ba9ef3f9b25306f42fde219e4b
I20260811 22:19:16.713603 11655 ts_manager.cc:194] Registered new tserver with Master: 6418cf43a4284bf397afb7296ab76b2b (127.29.229.2:36507)
I20260811 22:19:16.715057 11655 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:53135
I20260811 22:19:16.715824 12046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:16.696092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:40449 (local address 127.29.229.62:43617)
0811 22:19:16.696402 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:16.696411 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:16.697867 (+ 1456us) server_negotiation.cc:407] Connection header received
0811 22:19:16.698252 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:16.698260 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:16.698370 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:16.698486 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:16.700791 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:16.701793 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:16.703506 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:16.704276 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:16.704418 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:16.706788 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:16.706793 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:16.706797 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:16.706830 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:16.711378 (+ 4548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:16.713075 (+ 1697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:16.713081 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:16.713084 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:16.713478 (+ 394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:16.713980 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:16.713987 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:16.713991 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:16.714447 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:16.714820 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:16.715118 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 22:19:16.715397 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 22:19:16.716074 12198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:16.696721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43617 (local address 127.29.229.2:40449)
0811 22:19:16.697730 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:16.697746 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:16.698052 (+ 306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:16.698707 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:16.698715 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:16.699898 (+ 1183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:16.700527 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:16.700545 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:16.701947 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:16.701953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:16.703342 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:16.703358 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:16.704653 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:16.704659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:16.704872 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:16.705674 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:16.705693 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:16.706559 (+ 866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:16.711706 (+ 5147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:16.711715 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:16.711721 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:16.712761 (+ 1040us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:16.713637 (+ 876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:16.713642 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:16.713645 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:16.713834 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:16.714592 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:16.714597 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:16.714825 (+ 228us) client_negotiation.cc:769] Sending connection context
0811 22:19:16.715000 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 22:19:16.715393 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":873,"thread_start_us":744,"threads_started":1}
WARNING: no policy specified for oryx/127.29.229.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.3.keytab.
Entry for principal oryx/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.3.keytab.
I20260811 22:19:16.783696 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43617
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.3.keytab
--principal=oryx/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:38857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:19:17.078506 12005 heartbeater.cc:503] Master 127.29.229.62:43617 was elected leader, sending a full tablet report...
W20260811 22:19:17.147706 12224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:17.147997 12224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:17.148150 12224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:17.155555 12224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:17.155675 12224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:17.155823 12224 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:19:17.155913 12224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:19:17.166911 12224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:38857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.3.keytab
--principal=oryx/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.229.62:43617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:17.168363 12224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:17.169785 12224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:17.183840 12233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:17.183794 12235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:17.183893 12239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:17.188769 12224 server_base.cc:1034] running on GCE node
I20260811 22:19:17.189555 12224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:17.190671 12224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:17.191934 12224 hybrid_clock.cc:648] HybridClock initialized: now 1786486757191851 us; error 60 us; skew 500 ppm
I20260811 22:19:17.192281 12224 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:17 dist-test-slave-jq4t krb5kdc[11582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486757, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:17.200368 12224 init.cc:377] Logged in from keytab as oryx/127.29.229.3@KRBTEST.COM (short username oryx)
I20260811 22:19:17.201989 12224 webserver.cc:460] Webserver started at http://127.29.229.3:40099/ using document root <none> and password file <none>
I20260811 22:19:17.202729 12224 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:17.202822 12224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:17.203132 12224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:17.206323 12224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "20c48119b13c4c32a3f62a07698fd734"
format_stamp: "Formatted at 2026-08-11 22:19:17 on dist-test-slave-jq4t"
server_key: "6303df3659e48d9723286202e74584aa"
server_key_iv: "6663a7fd5334d0aabd4481e6965c251c"
server_key_version: "encryptionkey@0"
I20260811 22:19:17.207640 12224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "20c48119b13c4c32a3f62a07698fd734"
format_stamp: "Formatted at 2026-08-11 22:19:17 on dist-test-slave-jq4t"
server_key: "6303df3659e48d9723286202e74584aa"
server_key_iv: "6663a7fd5334d0aabd4481e6965c251c"
server_key_version: "encryptionkey@0"
I20260811 22:19:17.214397 12224 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.004s
I20260811 22:19:17.218384 12247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:17.219861 12224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:19:17.220057 12224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "20c48119b13c4c32a3f62a07698fd734"
format_stamp: "Formatted at 2026-08-11 22:19:17 on dist-test-slave-jq4t"
server_key: "6303df3659e48d9723286202e74584aa"
server_key_iv: "6663a7fd5334d0aabd4481e6965c251c"
server_key_version: "encryptionkey@0"
I20260811 22:19:17.220397 12224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:17.237779 12224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:17.242466 12224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:17.242779 12224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:17.245843 12224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:17.245962 12224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:17.246163 12224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:17.246254 12224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:19:17 dist-test-slave-jq4t krb5kdc[11582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486757, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM
I20260811 22:19:17.270092 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:17.246431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:39761 (local address 127.29.229.62:43617)
0811 22:19:17.247260 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:17.247267 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:17.248307 (+ 1040us) server_negotiation.cc:407] Connection header received
0811 22:19:17.250449 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:17.250458 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:17.250582 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:17.250726 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:17.253188 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:17.254356 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:17.256946 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:17.257532 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:17.257678 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:17.262456 (+ 4778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:17.262462 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:17.262465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:17.262499 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:17.265878 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:17.266920 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:17.266928 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:17.266933 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:17.267112 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:17.267716 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:17.267724 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:17.267728 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:17.268161 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:17.268360 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:17.269441 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0811 22:19:17.269676 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":705,"thread_start_us":552,"threads_started":1}
I20260811 22:19:17.271123 12259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:17.247117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43617 (local address 127.29.229.3:39761)
0811 22:19:17.248107 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:17.248159 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:17.250156 (+ 1997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:17.251042 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:17.251060 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:17.251942 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:17.252961 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:17.252996 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:17.254560 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:17.254569 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:17.256774 (+ 2205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:17.256787 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:17.257837 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:17.257850 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:17.258265 (+ 415us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:17.259179 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:17.259218 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:17.262204 (+ 2986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:17.266024 (+ 3820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:17.266038 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:17.266060 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:17.266731 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:17.267243 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:17.267248 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:17.267252 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:17.267549 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:17.268343 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:17.268357 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:17.268868 (+ 511us) client_negotiation.cc:769] Sending connection context
0811 22:19:17.269324 (+ 456us) client_negotiation.cc:240] Negotiation successful
0811 22:19:17.269919 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":450,"threads_started":1}
I20260811 22:19:17.303107 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:17.283804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:43375 (local address 127.29.229.62:43617)
0811 22:19:17.284226 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:17.284236 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:17.285023 (+ 787us) server_negotiation.cc:407] Connection header received
0811 22:19:17.285267 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:17.285277 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:17.285422 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:17.285579 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:17.286756 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:17.288038 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:17.290377 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:17.291436 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:17.291618 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:17.294609 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:17.294671 (+ 62us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:17.294684 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:17.294770 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:17.299571 (+ 4801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:17.300348 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:17.300357 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:17.300362 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:17.300602 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:17.301197 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:17.301204 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:17.301207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:17.301613 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:17.301768 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:17.302325 (+ 557us) server_negotiation.cc:299] Negotiation successful
0811 22:19:17.302618 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237}
I20260811 22:19:17.303102 12378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:17.283623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43617 (local address 127.29.229.3:43375)
0811 22:19:17.284854 (+ 1231us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:17.284880 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:17.285095 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:17.285653 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:17.285657 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:17.286280 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:17.286600 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:17.286611 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:17.288233 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:17.288263 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:17.290190 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:17.290222 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:17.291719 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:17.291730 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:17.292004 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:17.292858 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:17.292876 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:17.294176 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:17.299750 (+ 5574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:17.299754 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:17.299757 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:17.300201 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:17.300769 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:17.300775 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:17.300783 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:17.300996 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:17.301754 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:17.301758 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:17.301948 (+ 190us) client_negotiation.cc:769] Sending connection context
0811 22:19:17.302127 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 22:19:17.302521 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1012,"thread_start_us":801,"threads_started":1}
I20260811 22:19:17.303897 12224 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:33569
I20260811 22:19:17.303978 12436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:33569 every 8 connection(s)
I20260811 22:19:17.305476 12224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:19:17.305961 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 12224
I20260811 22:19:17.306147 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:19:17.306696 30612 external_mini_cluster.cc:1428] Setting key 4929f51c73cea7bd09024828cd6fae80
I20260811 22:19:17.308154 12437 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43617
I20260811 22:19:17.308498 12437 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:17.309553 12437 heartbeater.cc:511] Master 127.29.229.62:43617 requested a full tablet report, sending...
I20260811 22:19:17.311434 11654 ts_manager.cc:194] Registered new tserver with Master: 20c48119b13c4c32a3f62a07698fd734 (127.29.229.3:33569)
I20260811 22:19:17.313023 11654 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:39761
I20260811 22:19:17.322616 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:19:17 dist-test-slave-jq4t krb5kdc[11582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486757, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:19:17 dist-test-slave-jq4t krb5kdc[11582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:17 dist-test-slave-jq4t krb5kdc[11582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:17.719372 12219 heartbeater.cc:503] Master 127.29.229.62:43617 was elected leader, sending a full tablet report...
I20260811 22:19:18.315692 12437 heartbeater.cc:503] Master 127.29.229.62:43617 was elected leader, sending a full tablet report...
W20260811 22:19:18.540238 12449 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.143s user 0.000s sys 0.023s
W20260811 22:19:18.541517 12257 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:17.380418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50764 (local address 127.29.229.62:43617)
0811 22:19:17.380687 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:17.380697 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:17.389217 (+ 8520us) server_negotiation.cc:407] Connection header received
0811 22:19:17.389474 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:17.389481 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:17.389570 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:17.389646 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:17.391908 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:17.392883 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:17.395072 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:17.395696 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:17.395843 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:18.541255 (+1145412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50764: BlockingRecv error: recv got EOF from 127.0.0.1:50764 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 22:19:18 dist-test-slave-jq4t krb5kdc[11582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486757, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM
I20260811 22:19:18.561100 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:18.545903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50772 (local address 127.29.229.62:43617)
0811 22:19:18.546063 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:18.546068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:18.546465 (+ 397us) server_negotiation.cc:407] Connection header received
0811 22:19:18.546671 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:18.546675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:18.546759 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:18.546863 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:18.548107 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.548957 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.550189 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.550627 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.550736 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:18.555138 (+ 4402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:18.555143 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:18.555147 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:18.555184 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:18.558537 (+ 3353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:18.559267 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:18.559271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:18.559273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:18.559408 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:18.559740 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:18.559743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:18.559745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:18.560045 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:18.560253 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:18.560631 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 22:19:18.560850 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:19:18.565384 11654 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:19:18.573908 11654 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:19:18.609264 12473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:18.588943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57282 (local address 127.29.229.1:38279)
0811 22:19:18.590328 (+ 1385us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:18.590337 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:18.600215 (+ 9878us) server_negotiation.cc:407] Connection header received
0811 22:19:18.600728 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:18.600739 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:18.600943 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:18.601156 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:18.601902 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.603386 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.606297 (+ 2911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.607832 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.607996 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:18.608227 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:18.608563 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 22:19:18.608746 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1208,"thread_start_us":1229,"threads_started":1}
I20260811 22:19:18.609350 12470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:18.588076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:38279 (local address 127.0.0.1:57282)
0811 22:19:18.600063 (+ 11987us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:18.600086 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:18.600521 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:18.601134 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:18.601138 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:18.601165 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:18.601676 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:18.601693 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.603537 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.603544 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:18.606118 (+ 2574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:18.606133 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.608062 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.608068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:18.608261 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:18.608317 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 22:19:18.608456 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 22:19:18.608679 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11836,"mutex_wait_us":210,"thread_start_us":12172,"threads_started":1}
I20260811 22:19:18.609783 12466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:18.587635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:33569 (local address 127.0.0.1:45928)
0811 22:19:18.589276 (+ 1641us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:18.589319 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:18.589488 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:18.600866 (+ 11378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:18.600874 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:18.600923 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:18.601472 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:18.601487 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.603615 (+ 2128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.603620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:18.606817 (+ 3197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:18.606833 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.608831 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.608838 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:18.609073 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:18.609112 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 22:19:18.609179 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 22:19:18.609320 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1380,"mutex_wait_us":352,"thread_start_us":1205,"threads_started":1}
I20260811 22:19:18.609925 12474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:18.589686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:36507 (local address 127.0.0.1:38028)
0811 22:19:18.600468 (+ 10782us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:18.600534 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:18.600698 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:18.601514 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:18.601529 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:18.601557 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:18.602067 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:18.602082 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.603969 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.603973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:18.606817 (+ 2844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:18.606833 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.609101 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.609107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:18.609279 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:18.609306 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:19:18.609391 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 22:19:18.609538 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10675,"mutex_wait_us":334,"thread_start_us":10515,"threads_started":1}
I20260811 22:19:18.610787 12475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:18.590012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38028 (local address 127.29.229.2:36507)
0811 22:19:18.599695 (+ 9683us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:18.599708 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:18.600701 (+ 993us) server_negotiation.cc:407] Connection header received
0811 22:19:18.600847 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:18.600854 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:18.601084 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:18.601280 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:18.602294 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.603770 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.607028 (+ 3258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.608564 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.608719 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:18.608939 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:18.610187 (+ 1248us) server_negotiation.cc:299] Negotiation successful
0811 22:19:18.610415 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9451,"thread_start_us":9296,"threads_started":1}
I20260811 22:19:18.611274 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:18.588059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45928 (local address 127.29.229.3:33569)
0811 22:19:18.600064 (+ 12005us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:18.600074 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:18.600112 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:19:18.600233 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:18.600243 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:18.600598 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:18.601696 (+ 1098us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:18.601937 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.603490 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.607029 (+ 3539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.608564 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.610438 (+ 1874us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:18.610652 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:18.610723 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 22:19:18.610971 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11830,"thread_start_us":11686,"threads_started":1}
I20260811 22:19:18.614269 11885 tablet_service.cc:1467] Processing CreateTablet for tablet a37422e317d74a9884dc777a407a68d7 (DEFAULT_TABLE table=test-table [id=47f4116d3a4648b5b95012bc3cfb3558]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:18.614343 12319 tablet_service.cc:1467] Processing CreateTablet for tablet a37422e317d74a9884dc777a407a68d7 (DEFAULT_TABLE table=test-table [id=47f4116d3a4648b5b95012bc3cfb3558]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:18.615585 11885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a37422e317d74a9884dc777a407a68d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:18.615687 12319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a37422e317d74a9884dc777a407a68d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:18.617820 12108 tablet_service.cc:1467] Processing CreateTablet for tablet a37422e317d74a9884dc777a407a68d7 (DEFAULT_TABLE table=test-table [id=47f4116d3a4648b5b95012bc3cfb3558]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:18.619227 12108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a37422e317d74a9884dc777a407a68d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:18.636230 12476 tablet_bootstrap.cc:492] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae: Bootstrap starting.
I20260811 22:19:18.637399 12478 tablet_bootstrap.cc:492] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b: Bootstrap starting.
I20260811 22:19:18.640278 12477 tablet_bootstrap.cc:492] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734: Bootstrap starting.
I20260811 22:19:18.640443 12476 tablet_bootstrap.cc:654] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:18.641047 12478 tablet_bootstrap.cc:654] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:18.642510 12478 log.cc:824] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:18.642953 12476 log.cc:824] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:18.643970 12477 tablet_bootstrap.cc:654] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:18.645442 12477 log.cc:824] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:18.647699 12476 tablet_bootstrap.cc:492] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae: No bootstrap required, opened a new log
I20260811 22:19:18.648030 12476 ts_tablet_manager.cc:1397] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260811 22:19:18.648769 12477 tablet_bootstrap.cc:492] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734: No bootstrap required, opened a new log
I20260811 22:19:18.648890 12478 tablet_bootstrap.cc:492] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b: No bootstrap required, opened a new log
I20260811 22:19:18.649065 12477 ts_tablet_manager.cc:1397] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260811 22:19:18.649238 12478 ts_tablet_manager.cc:1397] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.004s
I20260811 22:19:18.656927 12478 raft_consensus.cc:348] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } }
I20260811 22:19:18.657378 12478 raft_consensus.cc:374] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:18.657524 12478 raft_consensus.cc:729] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6418cf43a4284bf397afb7296ab76b2b, State: Initialized, Role: FOLLOWER
I20260811 22:19:18.658274 12478 consensus_queue.cc:260] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } }
I20260811 22:19:18.660315 12476 raft_consensus.cc:348] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } }
I20260811 22:19:18.660892 12476 raft_consensus.cc:374] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:18.661055 12476 raft_consensus.cc:729] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db8de4f9d94255bde71c19700597ae, State: Initialized, Role: FOLLOWER
I20260811 22:19:18.661144 12478 ts_tablet_manager.cc:1428] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b: Time spent starting tablet: real 0.012s user 0.002s sys 0.009s
I20260811 22:19:18.661193 12477 raft_consensus.cc:348] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } }
I20260811 22:19:18.661684 12477 raft_consensus.cc:374] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:18.661820 12477 raft_consensus.cc:729] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c48119b13c4c32a3f62a07698fd734, State: Initialized, Role: FOLLOWER
I20260811 22:19:18.662012 12476 consensus_queue.cc:260] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } }
I20260811 22:19:18.662441 12477 consensus_queue.cc:260] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } }
I20260811 22:19:18.664381 12476 ts_tablet_manager.cc:1428] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260811 22:19:18.666162 12477 ts_tablet_manager.cc:1428] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734: Time spent starting tablet: real 0.017s user 0.010s sys 0.006s
W20260811 22:19:18.711061 12220 tablet.cc:2369] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:19:18.814460 12439 tablet.cc:2369] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:19:18.820801 12008 tablet.cc:2369] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:19:18.884512 12485 raft_consensus.cc:482] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:18.884903 12485 raft_consensus.cc:504] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } }
I20260811 22:19:18.886813 12485 leader_election.cc:288] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db8de4f9d94255bde71c19700597ae (127.29.229.1:38279), 6418cf43a4284bf397afb7296ab76b2b (127.29.229.2:36507)
I20260811 22:19:18.895545 12475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:18.887565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:48729 (local address 127.29.229.2:36507)
0811 22:19:18.888065 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:18.888074 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:18.889013 (+ 939us) server_negotiation.cc:407] Connection header received
0811 22:19:18.889354 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:18.889360 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:18.889460 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:18.889567 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:18.890422 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.891274 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.893817 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.894580 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.894776 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:18.894868 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:18.895150 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 22:19:18.895282 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209}
I20260811 22:19:18.895757 12487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:18.887506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:36507 (local address 127.29.229.3:48729)
0811 22:19:18.888860 (+ 1354us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:18.888879 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:18.889143 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:18.889669 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:18.889676 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:18.889715 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:18.890223 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:18.890237 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.891420 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.891428 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:18.893651 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:18.893665 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.894798 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.894805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:18.894985 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:18.895011 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:19:18.895123 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 22:19:18.895261 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1179,"mutex_wait_us":23,"spinlock_wait_cycles":32512,"thread_start_us":819,"threads_started":1}
I20260811 22:19:18.895819 12489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:18.887747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:38279 (local address 127.29.229.3:56443)
0811 22:19:18.888901 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:18.888917 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:18.889098 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:18.889543 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:18.889550 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:18.889588 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:18.890064 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:18.890084 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.891996 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.892002 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:18.893719 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:18.893732 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.894934 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.894940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:18.895118 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:18.895142 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:19:18.895250 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 22:19:18.895385 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1045,"mutex_wait_us":21,"thread_start_us":755,"threads_started":1}
I20260811 22:19:18.896080 12473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:18.887606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:56443 (local address 127.29.229.1:38279)
0811 22:19:18.887811 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:18.887820 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:18.889026 (+ 1206us) server_negotiation.cc:407] Connection header received
0811 22:19:18.889245 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:18.889255 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:18.889370 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:18.889456 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:18.890296 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.891816 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.893890 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.894700 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.894910 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:18.894984 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:18.895280 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 22:19:18.895413 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:19:18.897264 11927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a37422e317d74a9884dc777a407a68d7" candidate_uuid: "20c48119b13c4c32a3f62a07698fd734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db8de4f9d94255bde71c19700597ae" is_pre_election: true
I20260811 22:19:18.897264 12145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a37422e317d74a9884dc777a407a68d7" candidate_uuid: "20c48119b13c4c32a3f62a07698fd734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6418cf43a4284bf397afb7296ab76b2b" is_pre_election: true
I20260811 22:19:18.897900 12145 raft_consensus.cc:2454] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20c48119b13c4c32a3f62a07698fd734 in term 0.
I20260811 22:19:18.897900 11927 raft_consensus.cc:2454] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20c48119b13c4c32a3f62a07698fd734 in term 0.
I20260811 22:19:18.898761 12251 leader_election.cc:302] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 20c48119b13c4c32a3f62a07698fd734, 6418cf43a4284bf397afb7296ab76b2b, 99db8de4f9d94255bde71c19700597ae; no voters:
I20260811 22:19:18.899287 12485 raft_consensus.cc:2781] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:19:18.899480 12485 raft_consensus.cc:482] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:19:18.899600 12485 raft_consensus.cc:3037] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:18.902000 12485 raft_consensus.cc:504] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } }
I20260811 22:19:18.902674 12485 leader_election.cc:288] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [CANDIDATE]: Term 1 election: Requested vote from peers 99db8de4f9d94255bde71c19700597ae (127.29.229.1:38279), 6418cf43a4284bf397afb7296ab76b2b (127.29.229.2:36507)
I20260811 22:19:18.903226 11927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a37422e317d74a9884dc777a407a68d7" candidate_uuid: "20c48119b13c4c32a3f62a07698fd734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db8de4f9d94255bde71c19700597ae"
I20260811 22:19:18.903439 11927 raft_consensus.cc:3037] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:18.903627 12145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a37422e317d74a9884dc777a407a68d7" candidate_uuid: "20c48119b13c4c32a3f62a07698fd734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6418cf43a4284bf397afb7296ab76b2b"
I20260811 22:19:18.903844 12145 raft_consensus.cc:3037] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:18.905969 11927 raft_consensus.cc:2454] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c48119b13c4c32a3f62a07698fd734 in term 1.
I20260811 22:19:18.906423 12251 leader_election.cc:302] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c48119b13c4c32a3f62a07698fd734, 99db8de4f9d94255bde71c19700597ae; no voters:
I20260811 22:19:18.906682 12145 raft_consensus.cc:2454] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c48119b13c4c32a3f62a07698fd734 in term 1.
I20260811 22:19:18.906730 12485 raft_consensus.cc:2781] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:19:18.907306 12485 raft_consensus.cc:686] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [term 1 LEADER]: Becoming Leader. State: Replica: 20c48119b13c4c32a3f62a07698fd734, State: Running, Role: LEADER
I20260811 22:19:18.907848 12485 consensus_queue.cc:237] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } }
I20260811 22:19:18.912948 11653 catalog_manager.cc:5495] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20c48119b13c4c32a3f62a07698fd734 (127.29.229.3). New cstate: current_term: 1 leader_uuid: "20c48119b13c4c32a3f62a07698fd734" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } health_report { overall_health: UNKNOWN } } }
I20260811 22:19:18.978140 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:18.972439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45938 (local address 127.29.229.3:33569)
0811 22:19:18.972660 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:18.972665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:18.972739 (+ 74us) server_negotiation.cc:407] Connection header received
0811 22:19:18.973013 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:18.973017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:18.973119 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:18.973320 (+ 201us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:19:18.973668 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.974481 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.975886 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:18.976320 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:18.976431 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:18.976988 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:19:18.977557 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:19:18.977679 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:18.977830 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 22:19:18.977904 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 22:19:18.987249 12145 raft_consensus.cc:1261] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b [term 1 FOLLOWER]: Refusing update from remote peer 20c48119b13c4c32a3f62a07698fd734: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:18.987257 11927 raft_consensus.cc:1261] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae [term 1 FOLLOWER]: Refusing update from remote peer 20c48119b13c4c32a3f62a07698fd734: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:18.988435 12490 consensus_queue.cc:1035] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:18.988888 12485 consensus_queue.cc:1035] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:19.001297 12496 mvcc.cc:205] Tried to move back new op lower bound from 7317449764801294336 to 7317449764490698752. Current Snapshot: MvccSnapshot[applied={T|T < 7317449764801294336}]
I20260811 22:19:19.018236 11653 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:19.018576 11653 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:19.023145 11653 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847: Sending DeleteTablet for 3 replicas of tablet a37422e317d74a9884dc777a407a68d7
I20260811 22:19:19.024540 11885 tablet_service.cc:1514] Processing DeleteTablet for tablet a37422e317d74a9884dc777a407a68d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:19 UTC) from {username='oryx', principal='oryx/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:57282
I20260811 22:19:19.024566 12108 tablet_service.cc:1514] Processing DeleteTablet for tablet a37422e317d74a9884dc777a407a68d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:19 UTC) from {username='oryx', principal='oryx/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:38028
I20260811 22:19:19.025108 12319 tablet_service.cc:1514] Processing DeleteTablet for tablet a37422e317d74a9884dc777a407a68d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:19 UTC) from {username='oryx', principal='oryx/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:45928
I20260811 22:19:19.029212 11653 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:19:19.031246 11653 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:19:19.043349 12518 tablet_replica.cc:330] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734: stopping tablet replica
I20260811 22:19:19.044037 12107 tablet_service.cc:1467] Processing CreateTablet for tablet 30410e4d77c84e1cbc627d397503ac3b (DEFAULT_TABLE table=test-table [id=b688bd82ddac4d51a2f82fdbe771738e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:19.044879 12107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30410e4d77c84e1cbc627d397503ac3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:19.045956 12518 raft_consensus.cc:2229] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:19:19.046795 12516 tablet_replica.cc:330] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae: stopping tablet replica
I20260811 22:19:19.047238 12516 raft_consensus.cc:2229] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:19.047822 12516 raft_consensus.cc:2258] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:19.049692 11885 tablet_service.cc:1467] Processing CreateTablet for tablet 30410e4d77c84e1cbc627d397503ac3b (DEFAULT_TABLE table=test-table [id=b688bd82ddac4d51a2f82fdbe771738e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:19.049747 12319 tablet_service.cc:1467] Processing CreateTablet for tablet 30410e4d77c84e1cbc627d397503ac3b (DEFAULT_TABLE table=test-table [id=b688bd82ddac4d51a2f82fdbe771738e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:19.050558 11885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30410e4d77c84e1cbc627d397503ac3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:19.050559 12319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30410e4d77c84e1cbc627d397503ac3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:19.055687 12478 tablet_bootstrap.cc:492] T 30410e4d77c84e1cbc627d397503ac3b P 6418cf43a4284bf397afb7296ab76b2b: Bootstrap starting.
I20260811 22:19:19.057669 12478 tablet_bootstrap.cc:654] T 30410e4d77c84e1cbc627d397503ac3b P 6418cf43a4284bf397afb7296ab76b2b: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:19.057694 12476 tablet_bootstrap.cc:492] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae: Bootstrap starting.
I20260811 22:19:19.059405 12476 tablet_bootstrap.cc:654] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:19.061136 12478 tablet_bootstrap.cc:492] T 30410e4d77c84e1cbc627d397503ac3b P 6418cf43a4284bf397afb7296ab76b2b: No bootstrap required, opened a new log
I20260811 22:19:19.061383 12478 ts_tablet_manager.cc:1397] T 30410e4d77c84e1cbc627d397503ac3b P 6418cf43a4284bf397afb7296ab76b2b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:19:19.062285 12478 raft_consensus.cc:348] T 30410e4d77c84e1cbc627d397503ac3b P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:19.062981 12478 raft_consensus.cc:374] T 30410e4d77c84e1cbc627d397503ac3b P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:19.063139 12478 raft_consensus.cc:729] T 30410e4d77c84e1cbc627d397503ac3b P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6418cf43a4284bf397afb7296ab76b2b, State: Initialized, Role: FOLLOWER
I20260811 22:19:19.063527 12478 consensus_queue.cc:260] T 30410e4d77c84e1cbc627d397503ac3b P 6418cf43a4284bf397afb7296ab76b2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:19.064594 12478 ts_tablet_manager.cc:1428] T 30410e4d77c84e1cbc627d397503ac3b P 6418cf43a4284bf397afb7296ab76b2b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:19:19.063985 12518 raft_consensus.cc:2258] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:19.067479 12518 ts_tablet_manager.cc:1905] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:19.072358 12516 ts_tablet_manager.cc:1905] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:19.073822 12477 tablet_bootstrap.cc:492] T 30410e4d77c84e1cbc627d397503ac3b P 20c48119b13c4c32a3f62a07698fd734: Bootstrap starting.
I20260811 22:19:19.075078 12515 tablet_replica.cc:330] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b: stopping tablet replica
I20260811 22:19:19.075680 12477 tablet_bootstrap.cc:654] T 30410e4d77c84e1cbc627d397503ac3b P 20c48119b13c4c32a3f62a07698fd734: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:19.077651 12476 tablet_bootstrap.cc:492] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae: No bootstrap required, opened a new log
I20260811 22:19:19.077829 12476 ts_tablet_manager.cc:1397] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.006s
I20260811 22:19:19.078789 12476 raft_consensus.cc:348] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:19.079071 12476 raft_consensus.cc:374] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:19.079207 12476 raft_consensus.cc:729] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db8de4f9d94255bde71c19700597ae, State: Initialized, Role: FOLLOWER
I20260811 22:19:19.079609 12476 consensus_queue.cc:260] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:19.080461 12476 ts_tablet_manager.cc:1428] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260811 22:19:19.075541 12515 raft_consensus.cc:2229] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:19.085763 12515 raft_consensus.cc:2258] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:19.090294 12518 ts_tablet_manager.cc:1918] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:19.090586 12518 log.cc:1196] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/a37422e317d74a9884dc777a407a68d7
I20260811 22:19:19.091384 12518 ts_tablet_manager.cc:1939] T a37422e317d74a9884dc777a407a68d7 P 20c48119b13c4c32a3f62a07698fd734: Deleting consensus metadata
I20260811 22:19:19.094732 12477 tablet_bootstrap.cc:492] T 30410e4d77c84e1cbc627d397503ac3b P 20c48119b13c4c32a3f62a07698fd734: No bootstrap required, opened a new log
I20260811 22:19:19.094972 12477 ts_tablet_manager.cc:1397] T 30410e4d77c84e1cbc627d397503ac3b P 20c48119b13c4c32a3f62a07698fd734: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.003s
I20260811 22:19:19.094931 12515 ts_tablet_manager.cc:1905] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:19.096200 12516 ts_tablet_manager.cc:1918] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:19.096535 12516 log.cc:1196] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/wal/wals/a37422e317d74a9884dc777a407a68d7
I20260811 22:19:19.096743 12477 raft_consensus.cc:348] T 30410e4d77c84e1cbc627d397503ac3b P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:19.097065 12477 raft_consensus.cc:374] T 30410e4d77c84e1cbc627d397503ac3b P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:19.097406 12516 ts_tablet_manager.cc:1939] T a37422e317d74a9884dc777a407a68d7 P 99db8de4f9d94255bde71c19700597ae: Deleting consensus metadata
I20260811 22:19:19.097405 12477 raft_consensus.cc:729] T 30410e4d77c84e1cbc627d397503ac3b P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c48119b13c4c32a3f62a07698fd734, State: Initialized, Role: FOLLOWER
I20260811 22:19:19.097817 12477 consensus_queue.cc:260] T 30410e4d77c84e1cbc627d397503ac3b P 20c48119b13c4c32a3f62a07698fd734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:19.098510 11631 catalog_manager.cc:4841] TS 20c48119b13c4c32a3f62a07698fd734 (127.29.229.3:33569): tablet a37422e317d74a9884dc777a407a68d7 (table test-table [id=47f4116d3a4648b5b95012bc3cfb3558]) successfully deleted
I20260811 22:19:19.100672 11632 catalog_manager.cc:4841] TS 99db8de4f9d94255bde71c19700597ae (127.29.229.1:38279): tablet a37422e317d74a9884dc777a407a68d7 (table test-table [id=47f4116d3a4648b5b95012bc3cfb3558]) successfully deleted
I20260811 22:19:19.101475 12477 ts_tablet_manager.cc:1428] T 30410e4d77c84e1cbc627d397503ac3b P 20c48119b13c4c32a3f62a07698fd734: Time spent starting tablet: real 0.006s user 0.000s sys 0.004s
I20260811 22:19:19.103428 12515 ts_tablet_manager.cc:1918] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:19.103677 12515 log.cc:1196] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/a37422e317d74a9884dc777a407a68d7
I20260811 22:19:19.104283 12515 ts_tablet_manager.cc:1939] T a37422e317d74a9884dc777a407a68d7 P 6418cf43a4284bf397afb7296ab76b2b: Deleting consensus metadata
I20260811 22:19:19.106009 11631 catalog_manager.cc:4841] TS 6418cf43a4284bf397afb7296ab76b2b (127.29.229.2:36507): tablet a37422e317d74a9884dc777a407a68d7 (table test-table [id=47f4116d3a4648b5b95012bc3cfb3558]) successfully deleted
I20260811 22:19:19.205307 12484 raft_consensus.cc:482] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:19.205631 12484 raft_consensus.cc:504] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:19.207960 12484 leader_election.cc:288] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6418cf43a4284bf397afb7296ab76b2b (127.29.229.2:36507), 20c48119b13c4c32a3f62a07698fd734 (127.29.229.3:33569)
I20260811 22:19:19.216156 12475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.208642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:36251 (local address 127.29.229.2:36507)
0811 22:19:19.208853 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:19.208860 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:19.209716 (+ 856us) server_negotiation.cc:407] Connection header received
0811 22:19:19.209838 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:19.209846 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:19.209957 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:19.210072 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:19.210899 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.212098 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.214202 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.215031 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.215230 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.215317 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:19.215699 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 22:19:19.215852 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:19:19.216172 12525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.208637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:36507 (local address 127.29.229.1:36251)
0811 22:19:19.209396 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:19.209420 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:19.209568 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:19.210116 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:19.210122 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:19.210154 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:19.210658 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.210673 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.212243 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.212249 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.214036 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.214048 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.215267 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.215273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.215512 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.215547 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 22:19:19.215655 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 22:19:19.215791 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":420,"threads_started":1}
I20260811 22:19:19.217427 12145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30410e4d77c84e1cbc627d397503ac3b" candidate_uuid: "99db8de4f9d94255bde71c19700597ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6418cf43a4284bf397afb7296ab76b2b" is_pre_election: true
I20260811 22:19:19.217713 12145 raft_consensus.cc:2454] T 30410e4d77c84e1cbc627d397503ac3b P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db8de4f9d94255bde71c19700597ae in term 0.
I20260811 22:19:19.218173 12526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.209581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:33569 (local address 127.29.229.1:37645)
0811 22:19:19.210782 (+ 1201us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:19.210813 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:19.210963 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:19.211321 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:19.211325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:19.211349 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:19.211707 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.211719 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.213227 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.213233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.215394 (+ 2161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.215410 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.217229 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.217235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.217411 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.217433 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:19:19.217539 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 22:19:19.217754 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1067,"thread_start_us":845,"threads_started":1}
I20260811 22:19:19.218030 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.209732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:37645 (local address 127.29.229.3:33569)
0811 22:19:19.209896 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:19.209901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:19.210916 (+ 1015us) server_negotiation.cc:407] Connection header received
0811 22:19:19.211105 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:19.211108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:19.211167 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:19.211303 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:19.211901 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.213066 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.215619 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.216988 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.217203 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.217281 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:19.217577 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 22:19:19.217714 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:19:19.218407 11800 leader_election.cc:302] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6418cf43a4284bf397afb7296ab76b2b, 99db8de4f9d94255bde71c19700597ae; no voters:
I20260811 22:19:19.218864 12484 raft_consensus.cc:2781] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:19:19.219022 12484 raft_consensus.cc:482] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:19:19.219131 12484 raft_consensus.cc:3037] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:19.219655 12356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30410e4d77c84e1cbc627d397503ac3b" candidate_uuid: "99db8de4f9d94255bde71c19700597ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c48119b13c4c32a3f62a07698fd734" is_pre_election: true
I20260811 22:19:19.220175 12356 raft_consensus.cc:2454] T 30410e4d77c84e1cbc627d397503ac3b P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db8de4f9d94255bde71c19700597ae in term 0.
I20260811 22:19:19.221915 12484 raft_consensus.cc:504] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:19.222476 12484 leader_election.cc:288] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [CANDIDATE]: Term 1 election: Requested vote from peers 6418cf43a4284bf397afb7296ab76b2b (127.29.229.2:36507), 20c48119b13c4c32a3f62a07698fd734 (127.29.229.3:33569)
I20260811 22:19:19.223014 12145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30410e4d77c84e1cbc627d397503ac3b" candidate_uuid: "99db8de4f9d94255bde71c19700597ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6418cf43a4284bf397afb7296ab76b2b"
I20260811 22:19:19.223241 12145 raft_consensus.cc:3037] T 30410e4d77c84e1cbc627d397503ac3b P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:19.223425 12356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30410e4d77c84e1cbc627d397503ac3b" candidate_uuid: "99db8de4f9d94255bde71c19700597ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c48119b13c4c32a3f62a07698fd734"
I20260811 22:19:19.223675 12356 raft_consensus.cc:3037] T 30410e4d77c84e1cbc627d397503ac3b P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:19.226490 12145 raft_consensus.cc:2454] T 30410e4d77c84e1cbc627d397503ac3b P 6418cf43a4284bf397afb7296ab76b2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db8de4f9d94255bde71c19700597ae in term 1.
I20260811 22:19:19.226970 11800 leader_election.cc:302] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6418cf43a4284bf397afb7296ab76b2b, 99db8de4f9d94255bde71c19700597ae; no voters:
I20260811 22:19:19.227246 12356 raft_consensus.cc:2454] T 30410e4d77c84e1cbc627d397503ac3b P 20c48119b13c4c32a3f62a07698fd734 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db8de4f9d94255bde71c19700597ae in term 1.
I20260811 22:19:19.227365 12484 raft_consensus.cc:2781] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:19:19.227919 12484 raft_consensus.cc:686] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [term 1 LEADER]: Becoming Leader. State: Replica: 99db8de4f9d94255bde71c19700597ae, State: Running, Role: LEADER
I20260811 22:19:19.228387 12484 consensus_queue.cc:237] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:19.232244 11655 catalog_manager.cc:5495] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 99db8de4f9d94255bde71c19700597ae (127.29.229.1). New cstate: current_term: 1 leader_uuid: "99db8de4f9d94255bde71c19700597ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } health_report { overall_health: UNKNOWN } } }
Aug 11 22:19:19 dist-test-slave-jq4t krb5kdc[11582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486755, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.62@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM
I20260811 22:19:19.297261 12538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.279133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50782 (local address 127.29.229.62:43617)
0811 22:19:19.279977 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:19.279984 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:19.280336 (+ 352us) server_negotiation.cc:407] Connection header received
0811 22:19:19.280592 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:19.280596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:19.280661 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:19.280774 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:19.281856 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.282653 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.283926 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.284385 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.284554 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.289551 (+ 4997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:19.289564 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:19.289573 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:19.289652 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:19.293722 (+ 4070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:19.294867 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:19.294872 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:19.294875 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:19.295049 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:19.295513 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:19.295516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:19.295518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:19.295811 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:19.295959 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:19.296591 (+ 632us) server_negotiation.cc:299] Negotiation successful
0811 22:19:19.296933 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":126,"server-negotiator.queue_time_us":634,"thread_start_us":489,"threads_started":1}
I20260811 22:19:19.297284 12539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.279161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43617 (local address 127.0.0.1:50782)
0811 22:19:19.280216 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:19.280233 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:19.280417 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:19.280798 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:19.280803 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:19.281506 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:19.281705 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.281715 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.282793 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.282800 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.283780 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.283792 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.284587 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.284594 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.284790 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.286183 (+ 1393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:19.286205 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:19.289074 (+ 2869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:19.293936 (+ 4862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:19.293954 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:19.293962 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:19.294669 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:19.295183 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:19.295190 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:19.295193 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:19.295392 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:19.295955 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:19.295964 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:19.296175 (+ 211us) client_negotiation.cc:769] Sending connection context
0811 22:19:19.296388 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 22:19:19.296784 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":929,"thread_start_us":796,"threads_started":1}
I20260811 22:19:19.303285 11655 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:50782:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:19:19.304708 11655 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:19:19.313620 12319 tablet_service.cc:1467] Processing CreateTablet for tablet 2b1830bcf1b74559b9531c8ded579884 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c525ee6290b142b69a854797ff8d3559]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:19:19.313750 12108 tablet_service.cc:1467] Processing CreateTablet for tablet 2b1830bcf1b74559b9531c8ded579884 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c525ee6290b142b69a854797ff8d3559]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:19:19.314275 12319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b1830bcf1b74559b9531c8ded579884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:19.314352 12108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b1830bcf1b74559b9531c8ded579884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:19.319409 11885 tablet_service.cc:1467] Processing CreateTablet for tablet 2b1830bcf1b74559b9531c8ded579884 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c525ee6290b142b69a854797ff8d3559]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:19:19.320048 11885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b1830bcf1b74559b9531c8ded579884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:19.321911 12478 tablet_bootstrap.cc:492] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b: Bootstrap starting.
I20260811 22:19:19.323712 12478 tablet_bootstrap.cc:654] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:19.324627 12477 tablet_bootstrap.cc:492] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734: Bootstrap starting.
I20260811 22:19:19.326522 12477 tablet_bootstrap.cc:654] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:19.328022 12476 tablet_bootstrap.cc:492] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae: Bootstrap starting.
I20260811 22:19:19.329898 12476 tablet_bootstrap.cc:654] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:19.337976 12478 tablet_bootstrap.cc:492] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b: No bootstrap required, opened a new log
I20260811 22:19:19.338388 12478 ts_tablet_manager.cc:1397] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.004s
I20260811 22:19:19.339095 12477 tablet_bootstrap.cc:492] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734: No bootstrap required, opened a new log
I20260811 22:19:19.339332 12477 ts_tablet_manager.cc:1397] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.005s
I20260811 22:19:19.340083 12478 raft_consensus.cc:348] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } }
I20260811 22:19:19.341060 12478 raft_consensus.cc:374] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:19.340143 12477 raft_consensus.cc:348] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } }
I20260811 22:19:19.341370 12478 raft_consensus.cc:729] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6418cf43a4284bf397afb7296ab76b2b, State: Initialized, Role: FOLLOWER
I20260811 22:19:19.341562 12477 raft_consensus.cc:374] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:19.341717 12477 raft_consensus.cc:729] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c48119b13c4c32a3f62a07698fd734, State: Initialized, Role: FOLLOWER
I20260811 22:19:19.342041 12477 consensus_queue.cc:260] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } }
I20260811 22:19:19.342092 12478 consensus_queue.cc:260] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } }
I20260811 22:19:19.343060 12483 tablet_replica.cc:439] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } }
I20260811 22:19:19.343501 12483 tablet_replica.cc:442] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:19:19.342581 12508 tablet_replica.cc:439] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } }
I20260811 22:19:19.343801 12508 tablet_replica.cc:442] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:19:19.343986 12477 ts_tablet_manager.cc:1428] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260811 22:19:19.345481 12476 tablet_bootstrap.cc:492] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae: No bootstrap required, opened a new log
I20260811 22:19:19.345721 12476 ts_tablet_manager.cc:1397] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.005s
I20260811 22:19:19.346493 12478 ts_tablet_manager.cc:1428] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260811 22:19:19.346657 12476 raft_consensus.cc:348] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } }
I20260811 22:19:19.347370 12476 raft_consensus.cc:374] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:19.347518 12476 raft_consensus.cc:729] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db8de4f9d94255bde71c19700597ae, State: Initialized, Role: FOLLOWER
I20260811 22:19:19.347887 12476 consensus_queue.cc:260] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } }
I20260811 22:19:19.348436 12484 tablet_replica.cc:439] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } }
I20260811 22:19:19.348766 12484 tablet_replica.cc:442] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:19:19.349442 12476 ts_tablet_manager.cc:1428] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260811 22:19:19.562242 12483 raft_consensus.cc:482] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:19.562500 12483 raft_consensus.cc:504] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } }
I20260811 22:19:19.564073 12483 leader_election.cc:288] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db8de4f9d94255bde71c19700597ae (127.29.229.1:38279), 20c48119b13c4c32a3f62a07698fd734 (127.29.229.3:33569)
I20260811 22:19:19.564845 12508 raft_consensus.cc:482] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:19.565021 12508 raft_consensus.cc:504] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } }
I20260811 22:19:19.565768 12508 leader_election.cc:288] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db8de4f9d94255bde71c19700597ae (127.29.229.1:38279), 6418cf43a4284bf397afb7296ab76b2b (127.29.229.2:36507)
I20260811 22:19:19.566254 11927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b1830bcf1b74559b9531c8ded579884" candidate_uuid: "20c48119b13c4c32a3f62a07698fd734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db8de4f9d94255bde71c19700597ae" is_pre_election: true
I20260811 22:19:19.566481 12145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b1830bcf1b74559b9531c8ded579884" candidate_uuid: "20c48119b13c4c32a3f62a07698fd734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6418cf43a4284bf397afb7296ab76b2b" is_pre_election: true
I20260811 22:19:19.566541 11927 raft_consensus.cc:2454] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20c48119b13c4c32a3f62a07698fd734 in term 0.
I20260811 22:19:19.566738 12145 raft_consensus.cc:2454] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20c48119b13c4c32a3f62a07698fd734 in term 0.
I20260811 22:19:19.567034 12251 leader_election.cc:302] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c48119b13c4c32a3f62a07698fd734, 99db8de4f9d94255bde71c19700597ae; no voters:
I20260811 22:19:19.567344 12508 raft_consensus.cc:2781] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:19:19.567469 12508 raft_consensus.cc:482] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:19:19.567567 12508 raft_consensus.cc:3037] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:19.570668 12508 raft_consensus.cc:504] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } }
I20260811 22:19:19.571842 12145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b1830bcf1b74559b9531c8ded579884" candidate_uuid: "20c48119b13c4c32a3f62a07698fd734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6418cf43a4284bf397afb7296ab76b2b"
I20260811 22:19:19.571924 11927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b1830bcf1b74559b9531c8ded579884" candidate_uuid: "20c48119b13c4c32a3f62a07698fd734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db8de4f9d94255bde71c19700597ae"
I20260811 22:19:19.572098 12145 raft_consensus.cc:3037] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:19.572197 11927 raft_consensus.cc:3037] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:19.573256 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.564605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:35723 (local address 127.29.229.3:33569)
0811 22:19:19.564896 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:19.564904 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:19.565316 (+ 412us) server_negotiation.cc:407] Connection header received
0811 22:19:19.565565 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:19.565571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:19.565649 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:19.565753 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:19.566645 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.567794 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.570380 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.571666 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.571829 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.571913 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:19.572836 (+ 923us) server_negotiation.cc:299] Negotiation successful
0811 22:19:19.572991 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260811 22:19:19.574117 12546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.564535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:33569 (local address 127.29.229.2:35723)
0811 22:19:19.565180 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:19.565210 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:19.565397 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:19.565960 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:19.565965 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:19.565993 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:19.566418 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.566435 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.568001 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.568006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.570197 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.570213 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.572062 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.572068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.572617 (+ 549us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.572663 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 22:19:19.573494 (+ 831us) client_negotiation.cc:240] Negotiation successful
0811 22:19:19.573654 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":2,"thread_start_us":384,"threads_started":1}
I20260811 22:19:19.574268 12508 leader_election.cc:288] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [CANDIDATE]: Term 1 election: Requested vote from peers 99db8de4f9d94255bde71c19700597ae (127.29.229.1:38279), 6418cf43a4284bf397afb7296ab76b2b (127.29.229.2:36507)
I20260811 22:19:19.574683 12356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b1830bcf1b74559b9531c8ded579884" candidate_uuid: "6418cf43a4284bf397afb7296ab76b2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c48119b13c4c32a3f62a07698fd734" is_pre_election: true
I20260811 22:19:19.574988 12356 raft_consensus.cc:2379] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6418cf43a4284bf397afb7296ab76b2b in current term 1: Already voted for candidate 20c48119b13c4c32a3f62a07698fd734 in this term.
I20260811 22:19:19.575085 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.564554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:43113 (local address 127.29.229.1:38279)
0811 22:19:19.565872 (+ 1318us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:19.565878 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:19.566600 (+ 722us) server_negotiation.cc:407] Connection header received
0811 22:19:19.566744 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:19.566751 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:19.566890 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:19.567007 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:19.568051 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.569097 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.571102 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.572686 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.572869 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.572954 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:19.574435 (+ 1481us) server_negotiation.cc:299] Negotiation successful
0811 22:19:19.574580 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1181,"thread_start_us":3080,"threads_started":1}
I20260811 22:19:19.575387 11927 raft_consensus.cc:2454] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c48119b13c4c32a3f62a07698fd734 in term 1.
I20260811 22:19:19.575606 12548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.564576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:38279 (local address 127.29.229.2:43113)
0811 22:19:19.566195 (+ 1619us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:19.566212 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:19.566335 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:19.567235 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:19.567243 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:19.567273 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:19.567853 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.567868 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.569238 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.569244 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.570917 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.570933 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.573123 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.573130 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.573497 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.573541 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 22:19:19.573634 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 22:19:19.573786 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1480,"mutex_wait_us":23,"thread_start_us":782,"threads_started":1}
I20260811 22:19:19.575647 12145 raft_consensus.cc:2454] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c48119b13c4c32a3f62a07698fd734 in term 1.
I20260811 22:19:19.575843 12251 leader_election.cc:302] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c48119b13c4c32a3f62a07698fd734, 99db8de4f9d94255bde71c19700597ae; no voters:
I20260811 22:19:19.576200 12508 raft_consensus.cc:2781] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:19:19.576681 12508 raft_consensus.cc:686] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [term 1 LEADER]: Becoming Leader. State: Replica: 20c48119b13c4c32a3f62a07698fd734, State: Running, Role: LEADER
I20260811 22:19:19.576699 11927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b1830bcf1b74559b9531c8ded579884" candidate_uuid: "6418cf43a4284bf397afb7296ab76b2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db8de4f9d94255bde71c19700597ae" is_pre_election: true
I20260811 22:19:19.577033 12508 consensus_queue.cc:237] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } }
I20260811 22:19:19.577310 11927 raft_consensus.cc:2379] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6418cf43a4284bf397afb7296ab76b2b in current term 1: Already voted for candidate 20c48119b13c4c32a3f62a07698fd734 in this term.
I20260811 22:19:19.577859 12039 leader_election.cc:302] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6418cf43a4284bf397afb7296ab76b2b; no voters: 20c48119b13c4c32a3f62a07698fd734, 99db8de4f9d94255bde71c19700597ae
I20260811 22:19:19.578382 12483 raft_consensus.cc:2726] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 22:19:19.578969 12549 tablet_replica.cc:439] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734: TxnStatusTablet state changed. Reason: New leader 20c48119b13c4c32a3f62a07698fd734. Latest consensus state: current_term: 1 leader_uuid: "20c48119b13c4c32a3f62a07698fd734" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } }
I20260811 22:19:19.579341 12549 tablet_replica.cc:442] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:19:19.580116 12552 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:19:19.580348 12552 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:19:19.580755 11654 catalog_manager.cc:5495] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20c48119b13c4c32a3f62a07698fd734 (127.29.229.3). New cstate: current_term: 1 leader_uuid: "20c48119b13c4c32a3f62a07698fd734" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } health_report { overall_health: UNKNOWN } } }
I20260811 22:19:19.640925 12539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.635466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:33569 (local address 127.0.0.1:45954)
0811 22:19:19.635764 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:19.635785 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:19.635985 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:19.636395 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:19.636400 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:19.636412 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:19:19.636668 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.636678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.637728 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.637733 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.638765 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.638777 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.639626 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.639632 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.639815 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.639874 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:19:19.640290 (+ 416us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:19:19.640325 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 22:19:19.640428 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 22:19:19.640610 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 22:19:19.640930 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.635309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45954 (local address 127.29.229.3:33569)
0811 22:19:19.635562 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:19.635569 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:19.635874 (+ 305us) server_negotiation.cc:407] Connection header received
0811 22:19:19.636178 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:19.636184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:19.636262 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:19.636364 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:19:19.636819 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.637575 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.638919 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.639448 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.639597 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.640068 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:19:19.640182 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:19:19.640289 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:19.640465 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 22:19:19.640576 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:19:19.646204 12145 raft_consensus.cc:1261] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b [term 1 FOLLOWER]: Refusing update from remote peer 20c48119b13c4c32a3f62a07698fd734: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:19.646347 11927 raft_consensus.cc:1261] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae [term 1 FOLLOWER]: Refusing update from remote peer 20c48119b13c4c32a3f62a07698fd734: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:19.646907 12549 consensus_queue.cc:1035] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:19.647213 12508 consensus_queue.cc:1035] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:19.654345 12484 tablet_replica.cc:439] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae: TxnStatusTablet state changed. Reason: New leader 20c48119b13c4c32a3f62a07698fd734. Latest consensus state: current_term: 1 leader_uuid: "20c48119b13c4c32a3f62a07698fd734" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } }
I20260811 22:19:19.654680 12484 tablet_replica.cc:442] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:19:19.661237 12483 tablet_replica.cc:439] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b: TxnStatusTablet state changed. Reason: New leader 20c48119b13c4c32a3f62a07698fd734. Latest consensus state: current_term: 1 leader_uuid: "20c48119b13c4c32a3f62a07698fd734" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } }
I20260811 22:19:19.661623 12483 tablet_replica.cc:442] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:19:19.663897 12484 tablet_replica.cc:439] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "20c48119b13c4c32a3f62a07698fd734" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } }
I20260811 22:19:19.664208 12484 tablet_replica.cc:442] T 2b1830bcf1b74559b9531c8ded579884 P 99db8de4f9d94255bde71c19700597ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:19:19.672592 12508 tablet_replica.cc:439] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "20c48119b13c4c32a3f62a07698fd734" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } }
I20260811 22:19:19.673089 12508 tablet_replica.cc:442] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:19:19.676028 12567 tablet_replica.cc:439] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "20c48119b13c4c32a3f62a07698fd734" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } }
I20260811 22:19:19.676402 12567 tablet_replica.cc:442] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:19:19.678418 12483 tablet_replica.cc:439] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "20c48119b13c4c32a3f62a07698fd734" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } }
I20260811 22:19:19.678730 12483 tablet_replica.cc:442] T 2b1830bcf1b74559b9531c8ded579884 P 6418cf43a4284bf397afb7296ab76b2b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:19:19.685850 12484 consensus_queue.cc:1035] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:19.693711 12484 consensus_queue.cc:1035] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:19.696311 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.685597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57290 (local address 127.29.229.1:38279)
0811 22:19:19.685802 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:19.685809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:19.686454 (+ 645us) server_negotiation.cc:407] Connection header received
0811 22:19:19.686704 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:19.686710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:19.686817 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:19.686940 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:19:19.688633 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.689715 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.693334 (+ 3619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.694077 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.694214 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.694907 (+ 693us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:19:19.695627 (+ 720us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:19:19.695770 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:19.695912 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 22:19:19.696021 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:19:19.714555 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.707328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:50931 (local address 127.29.229.3:33569)
0811 22:19:19.707526 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:19.707534 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:19.709136 (+ 1602us) server_negotiation.cc:407] Connection header received
0811 22:19:19.709379 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:19.709383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:19.709451 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:19.709552 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:19:19.710095 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.710918 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.712522 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.713053 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.713176 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.713778 (+ 602us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:19:19.713889 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:19:19.713994 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:19.714145 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 22:19:19.714228 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:19:19.714648 12582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.707324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:33569 (local address 127.29.229.1:50931)
0811 22:19:19.709019 (+ 1695us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:19.709037 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:19.709223 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:19.709579 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:19.709584 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:19.709600 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:19:19.709932 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.709942 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.711115 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.711120 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.712352 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.712365 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.713336 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.713342 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.713515 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.713576 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:19:19.713997 (+ 421us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:19:19.714012 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 22:19:19.714114 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:19:19.714257 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1512,"thread_start_us":1362,"threads_started":1}
I20260811 22:19:19.774583 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.766698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:38291 (local address 127.29.229.1:38279)
0811 22:19:19.766940 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:19.766949 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:19.767692 (+ 743us) server_negotiation.cc:407] Connection header received
0811 22:19:19.768064 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:19.768070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:19.768196 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:19.768309 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:19:19.769116 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.770280 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.771913 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.772621 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.772770 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.773587 (+ 817us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:19:19.773765 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:19:19.773863 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:19.774085 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 22:19:19.774180 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:19:19.774703 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.766688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:38279 (local address 127.29.229.3:38291)
0811 22:19:19.767556 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:19.767579 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:19.767737 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:19.768344 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:19.768350 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:19.768363 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:19:19.768917 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.768932 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.770441 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.770453 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.771754 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:19.771766 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.772866 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.772874 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:19.773284 (+ 410us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.773381 (+ 97us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:19:19.773891 (+ 510us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:19:19.773904 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 22:19:19.774048 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 22:19:19.774210 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":466,"threads_started":1}
I20260811 22:19:19.822598 11654 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:19.822872 11654 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:19.826745 11654 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847: Sending DeleteTablet for 3 replicas of tablet 30410e4d77c84e1cbc627d397503ac3b
I20260811 22:19:19.827620 11885 tablet_service.cc:1514] Processing DeleteTablet for tablet 30410e4d77c84e1cbc627d397503ac3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:19 UTC) from {username='oryx', principal='oryx/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:57282
I20260811 22:19:19.827968 12108 tablet_service.cc:1514] Processing DeleteTablet for tablet 30410e4d77c84e1cbc627d397503ac3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:19 UTC) from {username='oryx', principal='oryx/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:38028
I20260811 22:19:19.828691 12319 tablet_service.cc:1514] Processing DeleteTablet for tablet 30410e4d77c84e1cbc627d397503ac3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:19 UTC) from {username='oryx', principal='oryx/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:45928
I20260811 22:19:19.836241 12597 tablet_replica.cc:330] T 30410e4d77c84e1cbc627d397503ac3b P 20c48119b13c4c32a3f62a07698fd734: stopping tablet replica
I20260811 22:19:19.836652 12597 raft_consensus.cc:2229] T 30410e4d77c84e1cbc627d397503ac3b P 20c48119b13c4c32a3f62a07698fd734 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:19.837092 12597 raft_consensus.cc:2258] T 30410e4d77c84e1cbc627d397503ac3b P 20c48119b13c4c32a3f62a07698fd734 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:19.838825 12597 ts_tablet_manager.cc:1905] T 30410e4d77c84e1cbc627d397503ac3b P 20c48119b13c4c32a3f62a07698fd734: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:19.854038 12597 ts_tablet_manager.cc:1918] T 30410e4d77c84e1cbc627d397503ac3b P 20c48119b13c4c32a3f62a07698fd734: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:19:19.854247 12597 log.cc:1196] T 30410e4d77c84e1cbc627d397503ac3b P 20c48119b13c4c32a3f62a07698fd734: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/30410e4d77c84e1cbc627d397503ac3b
I20260811 22:19:19.854763 12597 ts_tablet_manager.cc:1939] T 30410e4d77c84e1cbc627d397503ac3b P 20c48119b13c4c32a3f62a07698fd734: Deleting consensus metadata
I20260811 22:19:19.855063 12594 tablet_replica.cc:330] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae: stopping tablet replica
I20260811 22:19:19.855372 12594 raft_consensus.cc:2229] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:19:19.856299 11631 catalog_manager.cc:4841] TS 20c48119b13c4c32a3f62a07698fd734 (127.29.229.3:33569): tablet 30410e4d77c84e1cbc627d397503ac3b (table test-table [id=b688bd82ddac4d51a2f82fdbe771738e]) successfully deleted
I20260811 22:19:19.857020 12594 raft_consensus.cc:2258] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:19.857334 12595 tablet_replica.cc:330] T 30410e4d77c84e1cbc627d397503ac3b P 6418cf43a4284bf397afb7296ab76b2b: stopping tablet replica
I20260811 22:19:19.857564 12595 raft_consensus.cc:2229] T 30410e4d77c84e1cbc627d397503ac3b P 6418cf43a4284bf397afb7296ab76b2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:19.857889 12595 raft_consensus.cc:2258] T 30410e4d77c84e1cbc627d397503ac3b P 6418cf43a4284bf397afb7296ab76b2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:19.858872 12594 ts_tablet_manager.cc:1905] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:19.862784 12595 ts_tablet_manager.cc:1905] T 30410e4d77c84e1cbc627d397503ac3b P 6418cf43a4284bf397afb7296ab76b2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:19.867457 12594 ts_tablet_manager.cc:1918] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:19:19.867681 12594 log.cc:1196] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/wal/wals/30410e4d77c84e1cbc627d397503ac3b
I20260811 22:19:19.868250 12594 ts_tablet_manager.cc:1939] T 30410e4d77c84e1cbc627d397503ac3b P 99db8de4f9d94255bde71c19700597ae: Deleting consensus metadata
I20260811 22:19:19.869547 11632 catalog_manager.cc:4841] TS 99db8de4f9d94255bde71c19700597ae (127.29.229.1:38279): tablet 30410e4d77c84e1cbc627d397503ac3b (table test-table [id=b688bd82ddac4d51a2f82fdbe771738e]) successfully deleted
I20260811 22:19:19.870496 12595 ts_tablet_manager.cc:1918] T 30410e4d77c84e1cbc627d397503ac3b P 6418cf43a4284bf397afb7296ab76b2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:19:19.870651 12595 log.cc:1196] T 30410e4d77c84e1cbc627d397503ac3b P 6418cf43a4284bf397afb7296ab76b2b: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/30410e4d77c84e1cbc627d397503ac3b
I20260811 22:19:19.871133 12595 ts_tablet_manager.cc:1939] T 30410e4d77c84e1cbc627d397503ac3b P 6418cf43a4284bf397afb7296ab76b2b: Deleting consensus metadata
I20260811 22:19:19.872350 11631 catalog_manager.cc:4841] TS 6418cf43a4284bf397afb7296ab76b2b (127.29.229.2:36507): tablet 30410e4d77c84e1cbc627d397503ac3b (table test-table [id=b688bd82ddac4d51a2f82fdbe771738e]) successfully deleted
Aug 11 22:19:19 dist-test-slave-jq4t krb5kdc[11582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:19:19 dist-test-slave-jq4t krb5kdc[11582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:19 dist-test-slave-jq4t krb5kdc[11582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:19:19.969103 12610 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:19.935894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50794 (local address 127.29.229.62:43617)
0811 22:19:19.951323 (+ 15429us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:19.951331 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:19.952193 (+ 862us) server_negotiation.cc:407] Connection header received
0811 22:19:19.952500 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:19.952507 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:19.952600 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:19.952731 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:19.954762 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.956195 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.958198 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.958914 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.959052 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.968772 (+ 9720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50794: BlockingRecv error: recv got EOF from 127.0.0.1:50794 (error 108)
Metrics: {"server-negotiator.queue_time_us":15243,"thread_start_us":15447,"threads_started":1}
Aug 11 22:19:19 dist-test-slave-jq4t krb5kdc[11582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM
I20260811 22:19:20.001183 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:19.980996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50810 (local address 127.29.229.62:43617)
0811 22:19:19.981220 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:19.981228 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:19.981771 (+ 543us) server_negotiation.cc:407] Connection header received
0811 22:19:19.981991 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:19.981997 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:19.982087 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:19.982201 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:19.984005 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.985190 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.987189 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:19.987951 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:19.988112 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:19.993761 (+ 5649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:19.993769 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:19.993775 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:19.993828 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:19.997757 (+ 3929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:19.998712 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:19.998718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:19.998722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:19.998915 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:19.999412 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:19.999417 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:19.999420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:19.999808 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:20.000105 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:20.000474 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 22:19:20.000768 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:19:20.006007 11654 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50810:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:19:20.006793 11654 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:19:20.016949 11885 tablet_service.cc:1467] Processing CreateTablet for tablet 5a3c60e950584ae0a10db9b4761156f9 (DEFAULT_TABLE table=test-table [id=02420e5dd9ba48349377f5fae9b0798f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:20.017622 11885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3c60e950584ae0a10db9b4761156f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:20.017616 12108 tablet_service.cc:1467] Processing CreateTablet for tablet 5a3c60e950584ae0a10db9b4761156f9 (DEFAULT_TABLE table=test-table [id=02420e5dd9ba48349377f5fae9b0798f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:20.018141 12108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3c60e950584ae0a10db9b4761156f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:20.020788 12319 tablet_service.cc:1467] Processing CreateTablet for tablet 5a3c60e950584ae0a10db9b4761156f9 (DEFAULT_TABLE table=test-table [id=02420e5dd9ba48349377f5fae9b0798f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:20.021311 12319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3c60e950584ae0a10db9b4761156f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:20.036158 12623 tablet_bootstrap.cc:492] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae: Bootstrap starting.
I20260811 22:19:20.040884 12623 tablet_bootstrap.cc:654] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:20.047377 12627 tablet_bootstrap.cc:492] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734: Bootstrap starting.
I20260811 22:19:20.049059 12628 tablet_bootstrap.cc:492] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b: Bootstrap starting.
I20260811 22:19:20.050993 12628 tablet_bootstrap.cc:654] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:20.051642 12627 tablet_bootstrap.cc:654] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:20.052264 12623 tablet_bootstrap.cc:492] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae: No bootstrap required, opened a new log
I20260811 22:19:20.052546 12623 ts_tablet_manager.cc:1397] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.000s
I20260811 22:19:20.054481 12623 raft_consensus.cc:348] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:20.055385 12623 raft_consensus.cc:374] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:20.055590 12623 raft_consensus.cc:729] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db8de4f9d94255bde71c19700597ae, State: Initialized, Role: FOLLOWER
I20260811 22:19:20.056030 12623 consensus_queue.cc:260] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:20.057790 12627 tablet_bootstrap.cc:492] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734: No bootstrap required, opened a new log
I20260811 22:19:20.057950 12623 ts_tablet_manager.cc:1428] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 22:19:20.058066 12627 ts_tablet_manager.cc:1397] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.008s
I20260811 22:19:20.058951 12627 raft_consensus.cc:348] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:20.059290 12627 raft_consensus.cc:374] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:20.059459 12627 raft_consensus.cc:729] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c48119b13c4c32a3f62a07698fd734, State: Initialized, Role: FOLLOWER
I20260811 22:19:20.059908 12627 consensus_queue.cc:260] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:20.062723 12627 ts_tablet_manager.cc:1428] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260811 22:19:20.063409 12628 tablet_bootstrap.cc:492] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b: No bootstrap required, opened a new log
I20260811 22:19:20.063621 12628 ts_tablet_manager.cc:1397] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.000s
I20260811 22:19:20.064517 12628 raft_consensus.cc:348] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:20.064802 12628 raft_consensus.cc:374] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:20.064932 12628 raft_consensus.cc:729] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6418cf43a4284bf397afb7296ab76b2b, State: Initialized, Role: FOLLOWER
I20260811 22:19:20.065215 12628 consensus_queue.cc:260] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:20.065832 12628 ts_tablet_manager.cc:1428] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 22:19:20.277233 12483 raft_consensus.cc:482] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:20.277453 12483 raft_consensus.cc:504] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:20.278163 12483 leader_election.cc:288] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db8de4f9d94255bde71c19700597ae (127.29.229.1:38279), 20c48119b13c4c32a3f62a07698fd734 (127.29.229.3:33569)
I20260811 22:19:20.278681 11927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3c60e950584ae0a10db9b4761156f9" candidate_uuid: "6418cf43a4284bf397afb7296ab76b2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db8de4f9d94255bde71c19700597ae" is_pre_election: true
I20260811 22:19:20.278959 11927 raft_consensus.cc:2454] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6418cf43a4284bf397afb7296ab76b2b in term 0.
I20260811 22:19:20.278779 12356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3c60e950584ae0a10db9b4761156f9" candidate_uuid: "6418cf43a4284bf397afb7296ab76b2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c48119b13c4c32a3f62a07698fd734" is_pre_election: true
I20260811 22:19:20.279170 12356 raft_consensus.cc:2454] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6418cf43a4284bf397afb7296ab76b2b in term 0.
I20260811 22:19:20.279484 12039 leader_election.cc:302] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6418cf43a4284bf397afb7296ab76b2b, 99db8de4f9d94255bde71c19700597ae; no voters:
I20260811 22:19:20.279757 12483 raft_consensus.cc:2781] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:19:20.279891 12483 raft_consensus.cc:482] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:19:20.280035 12483 raft_consensus.cc:3037] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:20.282258 12483 raft_consensus.cc:504] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:20.282913 12483 leader_election.cc:288] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [CANDIDATE]: Term 1 election: Requested vote from peers 99db8de4f9d94255bde71c19700597ae (127.29.229.1:38279), 20c48119b13c4c32a3f62a07698fd734 (127.29.229.3:33569)
I20260811 22:19:20.283396 11927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3c60e950584ae0a10db9b4761156f9" candidate_uuid: "6418cf43a4284bf397afb7296ab76b2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db8de4f9d94255bde71c19700597ae"
I20260811 22:19:20.283456 12356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3c60e950584ae0a10db9b4761156f9" candidate_uuid: "6418cf43a4284bf397afb7296ab76b2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c48119b13c4c32a3f62a07698fd734"
I20260811 22:19:20.283633 11927 raft_consensus.cc:3037] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:20.283669 12356 raft_consensus.cc:3037] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:20.286728 12356 raft_consensus.cc:2454] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6418cf43a4284bf397afb7296ab76b2b in term 1.
I20260811 22:19:20.286936 11927 raft_consensus.cc:2454] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6418cf43a4284bf397afb7296ab76b2b in term 1.
I20260811 22:19:20.287352 12038 leader_election.cc:302] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c48119b13c4c32a3f62a07698fd734, 6418cf43a4284bf397afb7296ab76b2b; no voters:
I20260811 22:19:20.287760 12483 raft_consensus.cc:2781] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:19:20.288264 12483 raft_consensus.cc:686] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [term 1 LEADER]: Becoming Leader. State: Replica: 6418cf43a4284bf397afb7296ab76b2b, State: Running, Role: LEADER
I20260811 22:19:20.288749 12483 consensus_queue.cc:237] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:20.292984 11654 catalog_manager.cc:5495] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6418cf43a4284bf397afb7296ab76b2b (127.29.229.2). New cstate: current_term: 1 leader_uuid: "6418cf43a4284bf397afb7296ab76b2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } health_report { overall_health: UNKNOWN } } }
I20260811 22:19:20.349036 12637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.341972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38038 (local address 127.29.229.2:36507)
0811 22:19:20.342461 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:20.342465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:20.342482 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:19:20.342571 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:20.342574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:20.342651 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:20.342798 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:19:20.343260 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.344004 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.346145 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.346573 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.346703 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:20.347316 (+ 613us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:19:20.347918 (+ 602us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:19:20.348032 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:20.348318 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 22:19:20.348551 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":273,"threads_started":1}
I20260811 22:19:20.353657 11927 raft_consensus.cc:1261] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae [term 1 FOLLOWER]: Refusing update from remote peer 6418cf43a4284bf397afb7296ab76b2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:20.353989 12356 raft_consensus.cc:1261] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734 [term 1 FOLLOWER]: Refusing update from remote peer 6418cf43a4284bf397afb7296ab76b2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:20.354425 12633 consensus_queue.cc:1035] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:20.355198 12483 consensus_queue.cc:1035] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:20.363243 12640 mvcc.cc:205] Tried to move back new op lower bound from 7317449770400608256 to 7317449770148216832. Current Snapshot: MvccSnapshot[applied={T|T < 7317449770400608256}]
I20260811 22:19:20.393631 11654 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:20.393909 11654 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:20.398263 11654 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847: Sending DeleteTablet for 3 replicas of tablet 5a3c60e950584ae0a10db9b4761156f9
I20260811 22:19:20.401882 11885 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a3c60e950584ae0a10db9b4761156f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:20 UTC) from {username='oryx', principal='oryx/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:57282
I20260811 22:19:20.402304 12319 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a3c60e950584ae0a10db9b4761156f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:20 UTC) from {username='oryx', principal='oryx/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:45928
I20260811 22:19:20.405699 12108 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a3c60e950584ae0a10db9b4761156f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:20 UTC) from {username='oryx', principal='oryx/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:38028
I20260811 22:19:20.408126 11655 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50810:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:19:20.408914 11655 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:19:20.417125 12661 tablet_replica.cc:330] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b: stopping tablet replica
I20260811 22:19:20.417383 12661 raft_consensus.cc:2229] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:19:20.418018 12661 raft_consensus.cc:2258] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:20.418648 12657 tablet_replica.cc:330] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae: stopping tablet replica
I20260811 22:19:20.418876 12657 raft_consensus.cc:2229] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:20.419147 12657 raft_consensus.cc:2258] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:20.419865 12660 tablet_replica.cc:330] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734: stopping tablet replica
I20260811 22:19:20.420091 12660 raft_consensus.cc:2229] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:20.420380 12660 raft_consensus.cc:2258] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:20.421978 12661 ts_tablet_manager.cc:1905] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:20.421972 12660 ts_tablet_manager.cc:1905] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:20.421700 12657 ts_tablet_manager.cc:1905] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:20.423444 12108 tablet_service.cc:1467] Processing CreateTablet for tablet ba98726a41d2442e98c44bc324c63b14 (DEFAULT_TABLE table=test-table [id=1d221d164dfa48b3a6977bbb9dc2b579]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:20.423440 12319 tablet_service.cc:1467] Processing CreateTablet for tablet ba98726a41d2442e98c44bc324c63b14 (DEFAULT_TABLE table=test-table [id=1d221d164dfa48b3a6977bbb9dc2b579]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:20.423949 12108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba98726a41d2442e98c44bc324c63b14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:20.423942 12319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba98726a41d2442e98c44bc324c63b14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:20.429315 11885 tablet_service.cc:1467] Processing CreateTablet for tablet ba98726a41d2442e98c44bc324c63b14 (DEFAULT_TABLE table=test-table [id=1d221d164dfa48b3a6977bbb9dc2b579]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:20.429584 12660 ts_tablet_manager.cc:1918] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:20.429735 12660 log.cc:1196] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/5a3c60e950584ae0a10db9b4761156f9
I20260811 22:19:20.429821 11885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba98726a41d2442e98c44bc324c63b14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:20.430225 12660 ts_tablet_manager.cc:1939] T 5a3c60e950584ae0a10db9b4761156f9 P 20c48119b13c4c32a3f62a07698fd734: Deleting consensus metadata
I20260811 22:19:20.431459 11631 catalog_manager.cc:4841] TS 20c48119b13c4c32a3f62a07698fd734 (127.29.229.3:33569): tablet 5a3c60e950584ae0a10db9b4761156f9 (table test-table [id=02420e5dd9ba48349377f5fae9b0798f]) successfully deleted
I20260811 22:19:20.432214 12661 ts_tablet_manager.cc:1918] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:20.432358 12661 log.cc:1196] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/5a3c60e950584ae0a10db9b4761156f9
I20260811 22:19:20.433269 12661 ts_tablet_manager.cc:1939] T 5a3c60e950584ae0a10db9b4761156f9 P 6418cf43a4284bf397afb7296ab76b2b: Deleting consensus metadata
I20260811 22:19:20.433411 12628 tablet_bootstrap.cc:492] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b: Bootstrap starting.
I20260811 22:19:20.434458 11631 catalog_manager.cc:4841] TS 6418cf43a4284bf397afb7296ab76b2b (127.29.229.2:36507): tablet 5a3c60e950584ae0a10db9b4761156f9 (table test-table [id=02420e5dd9ba48349377f5fae9b0798f]) successfully deleted
I20260811 22:19:20.435281 12628 tablet_bootstrap.cc:654] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:20.436406 12627 tablet_bootstrap.cc:492] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734: Bootstrap starting.
I20260811 22:19:20.438122 12627 tablet_bootstrap.cc:654] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:20.438565 12623 tablet_bootstrap.cc:492] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae: Bootstrap starting.
I20260811 22:19:20.441548 12623 tablet_bootstrap.cc:654] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:20.449853 12623 tablet_bootstrap.cc:492] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae: No bootstrap required, opened a new log
I20260811 22:19:20.450102 12623 ts_tablet_manager.cc:1397] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260811 22:19:20.450299 12627 tablet_bootstrap.cc:492] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734: No bootstrap required, opened a new log
I20260811 22:19:20.450541 12627 ts_tablet_manager.cc:1397] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.010s
I20260811 22:19:20.451087 12623 raft_consensus.cc:348] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:20.451400 12623 raft_consensus.cc:374] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:20.451498 12623 raft_consensus.cc:729] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db8de4f9d94255bde71c19700597ae, State: Initialized, Role: FOLLOWER
I20260811 22:19:20.451543 12627 raft_consensus.cc:348] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:20.451771 12623 consensus_queue.cc:260] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:20.451915 12627 raft_consensus.cc:374] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:20.452075 12627 raft_consensus.cc:729] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c48119b13c4c32a3f62a07698fd734, State: Initialized, Role: FOLLOWER
I20260811 22:19:20.452364 12623 ts_tablet_manager.cc:1428] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:19:20.452415 12627 consensus_queue.cc:260] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:20.452750 12657 ts_tablet_manager.cc:1918] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:20.452911 12657 log.cc:1196] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/wal/wals/5a3c60e950584ae0a10db9b4761156f9
I20260811 22:19:20.453444 12657 ts_tablet_manager.cc:1939] T 5a3c60e950584ae0a10db9b4761156f9 P 99db8de4f9d94255bde71c19700597ae: Deleting consensus metadata
I20260811 22:19:20.455282 12628 tablet_bootstrap.cc:492] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b: No bootstrap required, opened a new log
I20260811 22:19:20.455513 12628 ts_tablet_manager.cc:1397] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.007s
I20260811 22:19:20.456033 11632 catalog_manager.cc:4841] TS 99db8de4f9d94255bde71c19700597ae (127.29.229.1:38279): tablet 5a3c60e950584ae0a10db9b4761156f9 (table test-table [id=02420e5dd9ba48349377f5fae9b0798f]) successfully deleted
I20260811 22:19:20.456460 12628 raft_consensus.cc:348] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:20.456751 12628 raft_consensus.cc:374] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:20.456866 12628 raft_consensus.cc:729] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6418cf43a4284bf397afb7296ab76b2b, State: Initialized, Role: FOLLOWER
I20260811 22:19:20.457116 12628 consensus_queue.cc:260] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:20.457751 12628 ts_tablet_manager.cc:1428] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 22:19:20.458264 12627 ts_tablet_manager.cc:1428] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734: Time spent starting tablet: real 0.008s user 0.000s sys 0.001s
I20260811 22:19:20.711354 12571 raft_consensus.cc:482] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:20.711588 12571 raft_consensus.cc:504] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:20.712415 12571 leader_election.cc:288] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db8de4f9d94255bde71c19700597ae (127.29.229.1:38279), 6418cf43a4284bf397afb7296ab76b2b (127.29.229.2:36507)
I20260811 22:19:20.713145 11927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba98726a41d2442e98c44bc324c63b14" candidate_uuid: "20c48119b13c4c32a3f62a07698fd734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db8de4f9d94255bde71c19700597ae" is_pre_election: true
I20260811 22:19:20.713186 12144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba98726a41d2442e98c44bc324c63b14" candidate_uuid: "20c48119b13c4c32a3f62a07698fd734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6418cf43a4284bf397afb7296ab76b2b" is_pre_election: true
I20260811 22:19:20.713452 11927 raft_consensus.cc:2454] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20c48119b13c4c32a3f62a07698fd734 in term 0.
I20260811 22:19:20.713529 12144 raft_consensus.cc:2454] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20c48119b13c4c32a3f62a07698fd734 in term 0.
I20260811 22:19:20.714109 12250 leader_election.cc:302] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c48119b13c4c32a3f62a07698fd734, 6418cf43a4284bf397afb7296ab76b2b; no voters:
I20260811 22:19:20.714460 12571 raft_consensus.cc:2781] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:19:20.714583 12571 raft_consensus.cc:482] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:19:20.714668 12571 raft_consensus.cc:3037] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:20.717795 12571 raft_consensus.cc:504] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:20.718916 11927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba98726a41d2442e98c44bc324c63b14" candidate_uuid: "20c48119b13c4c32a3f62a07698fd734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db8de4f9d94255bde71c19700597ae"
I20260811 22:19:20.718942 12144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba98726a41d2442e98c44bc324c63b14" candidate_uuid: "20c48119b13c4c32a3f62a07698fd734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6418cf43a4284bf397afb7296ab76b2b"
I20260811 22:19:20.719144 11927 raft_consensus.cc:3037] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:20.719168 12144 raft_consensus.cc:3037] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:20.722522 11927 raft_consensus.cc:2454] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c48119b13c4c32a3f62a07698fd734 in term 1.
I20260811 22:19:20.722781 12144 raft_consensus.cc:2454] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c48119b13c4c32a3f62a07698fd734 in term 1.
I20260811 22:19:20.723213 12250 leader_election.cc:302] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c48119b13c4c32a3f62a07698fd734, 6418cf43a4284bf397afb7296ab76b2b; no voters:
I20260811 22:19:20.724687 12571 leader_election.cc:288] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [CANDIDATE]: Term 1 election: Requested vote from peers 99db8de4f9d94255bde71c19700597ae (127.29.229.1:38279), 6418cf43a4284bf397afb7296ab76b2b (127.29.229.2:36507)
I20260811 22:19:20.724890 12571 raft_consensus.cc:2781] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:19:20.725035 12571 raft_consensus.cc:686] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [term 1 LEADER]: Becoming Leader. State: Replica: 20c48119b13c4c32a3f62a07698fd734, State: Running, Role: LEADER
I20260811 22:19:20.725343 12571 consensus_queue.cc:237] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } }
I20260811 22:19:20.730450 11654 catalog_manager.cc:5495] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20c48119b13c4c32a3f62a07698fd734 (127.29.229.3). New cstate: current_term: 1 leader_uuid: "20c48119b13c4c32a3f62a07698fd734" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20c48119b13c4c32a3f62a07698fd734" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 33569 } health_report { overall_health: HEALTHY } } }
I20260811 22:19:20.821847 12678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.815161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45956 (local address 127.29.229.3:33569)
0811 22:19:20.815819 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:20.815823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:20.815841 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:19:20.815891 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:20.815894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:20.815955 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:20.816067 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:19:20.816535 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.817352 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.819284 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.819934 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.820175 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:20.820771 (+ 596us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:19:20.820900 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:19:20.821023 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:20.821485 (+ 462us) server_negotiation.cc:299] Negotiation successful
0811 22:19:20.821574 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":310,"threads_started":1}
I20260811 22:19:20.837280 12678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.831436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:43033 (local address 127.29.229.3:33569)
0811 22:19:20.831632 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:20.831639 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:20.831985 (+ 346us) server_negotiation.cc:407] Connection header received
0811 22:19:20.832129 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:20.832134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:20.832215 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:20.832312 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:19:20.832809 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.833864 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.835206 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.835857 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.835984 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:20.836456 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:19:20.836615 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:19:20.836702 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:20.836906 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 22:19:20.836985 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 22:19:20.837332 12681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.831162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:33569 (local address 127.29.229.3:43033)
0811 22:19:20.831884 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:20.831901 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:20.832018 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:20.832334 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:20.832337 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:20.832347 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:19:20.832654 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:20.832664 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.834035 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.834040 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:20.835053 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:20.835065 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.836043 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.836047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:20.836257 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:20.836296 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:19:20.836748 (+ 452us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:19:20.836760 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 22:19:20.836853 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 22:19:20.836961 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":401,"threads_started":1}
I20260811 22:19:20.847254 12144 raft_consensus.cc:1261] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b [term 1 FOLLOWER]: Refusing update from remote peer 20c48119b13c4c32a3f62a07698fd734: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:20.847338 11927 raft_consensus.cc:1261] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae [term 1 FOLLOWER]: Refusing update from remote peer 20c48119b13c4c32a3f62a07698fd734: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:20.847900 12671 consensus_queue.cc:1035] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6418cf43a4284bf397afb7296ab76b2b" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 36507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:20.848443 12671 consensus_queue.cc:1035] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db8de4f9d94255bde71c19700597ae" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 38279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:20.906356 11654 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:20.906796 11654 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:20.910190 11654 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04f4c66999bd4c0cb9e040becfb76847: Sending DeleteTablet for 3 replicas of tablet ba98726a41d2442e98c44bc324c63b14
I20260811 22:19:20.911362 11885 tablet_service.cc:1514] Processing DeleteTablet for tablet ba98726a41d2442e98c44bc324c63b14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:20 UTC) from {username='oryx', principal='oryx/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:57282
I20260811 22:19:20.911427 12108 tablet_service.cc:1514] Processing DeleteTablet for tablet ba98726a41d2442e98c44bc324c63b14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:20 UTC) from {username='oryx', principal='oryx/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:38028
I20260811 22:19:20.911446 12318 tablet_service.cc:1514] Processing DeleteTablet for tablet ba98726a41d2442e98c44bc324c63b14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:20 UTC) from {username='oryx', principal='oryx/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:45928
I20260811 22:19:20.911695 12661 tablet_replica.cc:330] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b: stopping tablet replica
I20260811 22:19:20.911712 12657 tablet_replica.cc:330] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae: stopping tablet replica
I20260811 22:19:20.911765 12660 tablet_replica.cc:330] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734: stopping tablet replica
I20260811 22:19:20.911911 12661 raft_consensus.cc:2229] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:20.911929 12657 raft_consensus.cc:2229] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:20.911958 12660 raft_consensus.cc:2229] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:19:20.912137 12661 raft_consensus.cc:2258] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:20.912133 12657 raft_consensus.cc:2258] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:20.912328 12660 raft_consensus.cc:2258] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:20.913432 12661 ts_tablet_manager.cc:1905] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:20.913503 12657 ts_tablet_manager.cc:1905] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:20.913540 12660 ts_tablet_manager.cc:1905] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:20.918789 12661 ts_tablet_manager.cc:1918] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:19:20.918972 12661 log.cc:1196] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/ba98726a41d2442e98c44bc324c63b14
I20260811 22:19:20.919375 12661 ts_tablet_manager.cc:1939] T ba98726a41d2442e98c44bc324c63b14 P 6418cf43a4284bf397afb7296ab76b2b: Deleting consensus metadata
I20260811 22:19:20.920346 12660 ts_tablet_manager.cc:1918] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:19:20.920385 12657 ts_tablet_manager.cc:1918] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:19:20.920536 12660 log.cc:1196] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/ba98726a41d2442e98c44bc324c63b14
I20260811 22:19:20.920511 11631 catalog_manager.cc:4841] TS 6418cf43a4284bf397afb7296ab76b2b (127.29.229.2:36507): tablet ba98726a41d2442e98c44bc324c63b14 (table test-table [id=1d221d164dfa48b3a6977bbb9dc2b579]) successfully deleted
I20260811 22:19:20.920603 12657 log.cc:1196] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486685119861-30612-0/minicluster-data/ts-0/wal/wals/ba98726a41d2442e98c44bc324c63b14
I20260811 22:19:20.921052 12657 ts_tablet_manager.cc:1939] T ba98726a41d2442e98c44bc324c63b14 P 99db8de4f9d94255bde71c19700597ae: Deleting consensus metadata
I20260811 22:19:20.921253 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 11765
I20260811 22:19:20.922201 11632 catalog_manager.cc:4841] TS 99db8de4f9d94255bde71c19700597ae (127.29.229.1:38279): tablet ba98726a41d2442e98c44bc324c63b14 (table test-table [id=1d221d164dfa48b3a6977bbb9dc2b579]) successfully deleted
I20260811 22:19:20.923094 12660 ts_tablet_manager.cc:1939] T ba98726a41d2442e98c44bc324c63b14 P 20c48119b13c4c32a3f62a07698fd734: Deleting consensus metadata
I20260811 22:19:20.924011 11631 catalog_manager.cc:4841] TS 20c48119b13c4c32a3f62a07698fd734 (127.29.229.3:33569): tablet ba98726a41d2442e98c44bc324c63b14 (table test-table [id=1d221d164dfa48b3a6977bbb9dc2b579]) successfully deleted
Aug 11 22:19:20 dist-test-slave-jq4t krb5kdc[11582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.229.1@KRBTEST.COM
I20260811 22:19:20.937364 12694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:20.922471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57292 (local address 127.29.229.1:38279)
0811 22:19:20.922913 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:20.922921 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:20.923001 (+ 80us) server_negotiation.cc:407] Connection header received
0811 22:19:20.923217 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:20.923223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:20.923292 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:20.923415 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:20.924850 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.925652 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.926861 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:20.927274 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:20.927413 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:20.931668 (+ 4255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:20.931682 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:20.931698 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:20.931728 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:20.934501 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:20.935174 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:20.935180 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:20.935182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:20.935301 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:20.935685 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:20.935690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:20.935693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:20.936058 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:20.936395 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:20.936708 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 22:19:20.936968 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":212,"threads_started":1}
I20260811 22:19:20.938050 11998 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:21.071449 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 11765
I20260811 22:19:21.120030 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 12012
Aug 11 22:19:21 dist-test-slave-jq4t krb5kdc[11582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.229.2@KRBTEST.COM
I20260811 22:19:21.137835 12696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:21.120904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38040 (local address 127.29.229.2:36507)
0811 22:19:21.121458 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:21.121465 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:21.121488 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:19:21.121563 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:21.121567 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:21.121629 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:21.121725 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:21.123716 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:21.124624 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:21.126234 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:21.126811 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:21.126977 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:21.131488 (+ 4511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:21.131509 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:21.131527 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:21.131578 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:21.134747 (+ 3169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:21.135480 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:21.135487 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:21.135490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:21.135631 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:21.136079 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:21.136083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:21.136085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:21.136575 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:21.136831 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:21.137227 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 22:19:21.137485 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":303,"threads_started":1}
I20260811 22:19:21.138577 12213 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:21.275043 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 12012
I20260811 22:19:21.283679 12698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:21.282402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:38279 (local address 127.29.229.3:40963)
0811 22:19:21.283315 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:21.283433 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:38279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":679,"threads_started":1}
W20260811 22:19:21.283980 12251 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.229.1:38279: connect: Connection refused (error 111)
I20260811 22:19:21.285604 12698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:21.285193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:38279 (local address 127.29.229.3:48493)
0811 22:19:21.285417 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:21.285488 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:38279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 22:19:21.285797 12251 consensus_peers.cc:487] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 -> Peer 99db8de4f9d94255bde71c19700597ae (127.29.229.1:38279): Couldn't send request to peer 99db8de4f9d94255bde71c19700597ae. Status: Network error: Client connection negotiation failed: client connection to 127.29.229.1:38279: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 22:19:21.312656 12250 connection.cc:535] client connection to 127.29.229.2:36507 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:19:21.313367 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 12224
I20260811 22:19:21.314111 12698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:21.313606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:36507 (local address 127.29.229.3:45569)
0811 22:19:21.313786 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:21.313918 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.2:36507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 22:19:21.314538 12250 consensus_peers.cc:487] T 2b1830bcf1b74559b9531c8ded579884 P 20c48119b13c4c32a3f62a07698fd734 -> Peer 6418cf43a4284bf397afb7296ab76b2b (127.29.229.2:36507): Couldn't send request to peer 6418cf43a4284bf397afb7296ab76b2b. Status: Network error: Client connection negotiation failed: client connection to 127.29.229.2:36507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Aug 11 22:19:21 dist-test-slave-jq4t krb5kdc[11582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.229.3@KRBTEST.COM
I20260811 22:19:21.331878 12678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:21.314383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45958 (local address 127.29.229.3:33569)
0811 22:19:21.314576 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:21.314583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:21.314605 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:19:21.314673 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:21.314678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:21.314768 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:21.314856 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:21.316976 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:21.317809 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:21.319628 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:21.320229 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:21.320402 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:21.325336 (+ 4934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:21.325355 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:21.325373 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:21.325423 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:21.329003 (+ 3580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:21.329741 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:21.329748 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:21.329751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:21.329891 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:21.330272 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:21.330277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:21.330282 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:21.330664 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:21.331083 (+ 419us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:21.331278 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 22:19:21.331504 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:19:21.332640 12429 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:21.507778 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 12224
I20260811 22:19:21.554797 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 11604
I20260811 22:19:21.556124 11710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:21.679730 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 11604
2026-08-11T22:19:21Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (8566 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:19:21 dist-test-slave-jq4t krb5kdc[12705](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:19:21 dist-test-slave-jq4t krb5kdc[12705](info): set up 2 sockets
Aug 11 22:19:21 dist-test-slave-jq4t krb5kdc[12705](info): commencing operation
krb5kdc: starting...
W20260811 22:19:23.093180 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.350s user 0.003s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:19:23 dist-test-slave-jq4t krb5kdc[12705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:19:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:19:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:19:23.424827 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:44643
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:35483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:44643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:23.791741 12726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:23.792141 12726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:23.792321 12726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:23.799700 12726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:19:23.799885 12726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:23.800005 12726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:23.800115 12726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:19:23.800209 12726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:19:23.810859 12726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:35483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:44643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:44643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:23.812527 12726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:23.813997 12726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:23.827261 12733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:23.827628 12726 server_base.cc:1034] running on GCE node
W20260811 22:19:23.827917 12737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:23.827966 12731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:23.829025 12726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:23.830576 12726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:23.831857 12726 hybrid_clock.cc:648] HybridClock initialized: now 1786486763831785 us; error 59 us; skew 500 ppm
I20260811 22:19:23.832278 12726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T22:19:23Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2217 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:19:24 dist-test-slave-jq4t krb5kdc[12749](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:19:24 dist-test-slave-jq4t krb5kdc[12749](info): set up 2 sockets
Aug 11 22:19:24 dist-test-slave-jq4t krb5kdc[12749](info): commencing operation
krb5kdc: starting...
W20260811 22:19:25.329157 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.367s user 0.007s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:19:25 dist-test-slave-jq4t krb5kdc[12749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:19:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:19:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:19:25.672796 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:34795
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:34795
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:26.035120 12765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:26.035499 12765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:26.035696 12765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:26.043118 12765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:19:26.043236 12765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:26.043284 12765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:26.043347 12765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:19:26.043394 12765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:19:26.053288 12765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:34795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:34795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:26.054538 12765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:26.056010 12765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:26.069996 12775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:26.069985 12776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:26.069916 12780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:26.070983 12765 server_base.cc:1034] running on GCE node
I20260811 22:19:26.071928 12765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:26.073592 12765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:26.074836 12765 hybrid_clock.cc:648] HybridClock initialized: now 1786486766074790 us; error 45 us; skew 500 ppm
I20260811 22:19:26.075209 12765 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:26 dist-test-slave-jq4t krb5kdc[12749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486766, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:26.078588 12765 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:19:26.080159 12765 webserver.cc:460] Webserver started at http://127.29.229.62:42069/ using document root <none> and password file <none>
I20260811 22:19:26.080952 12765 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:26.081106 12765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:26.081439 12765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:26.084637 12765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59"
format_stamp: "Formatted at 2026-08-11 22:19:26 on dist-test-slave-jq4t"
server_key: "17c68564a144f6cd53409f6c0fb1be90"
server_key_iv: "114bba7eeeb39f0c6f523feca2d5d46c"
server_key_version: "encryptionkey@0"
I20260811 22:19:26.085500 12765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59"
format_stamp: "Formatted at 2026-08-11 22:19:26 on dist-test-slave-jq4t"
server_key: "17c68564a144f6cd53409f6c0fb1be90"
server_key_iv: "114bba7eeeb39f0c6f523feca2d5d46c"
server_key_version: "encryptionkey@0"
I20260811 22:19:26.091394 12765 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 22:19:26.095896 12791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:26.098136 12765 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:19:26.098480 12765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59"
format_stamp: "Formatted at 2026-08-11 22:19:26 on dist-test-slave-jq4t"
server_key: "17c68564a144f6cd53409f6c0fb1be90"
server_key_iv: "114bba7eeeb39f0c6f523feca2d5d46c"
server_key_version: "encryptionkey@0"
I20260811 22:19:26.098758 12765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:26.122058 12765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:26.126920 12765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:26.127272 12765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:26.148996 12765 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:34795
I20260811 22:19:26.149036 12875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:34795 every 8 connection(s)
I20260811 22:19:26.150503 12765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:19:26.154429 12877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:26.155782 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 12765
I20260811 22:19:26.156024 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:19:26.156775 30612 external_mini_cluster.cc:1428] Setting key 3decaf4e8b6edce7796ab546259b94ba
I20260811 22:19:26.170576 12877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59: Bootstrap starting.
Aug 11 22:19:26 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:19:26.175180 12877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:26.177064 12877 log.cc:824] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:26.180388 12877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59: No bootstrap required, opened a new log
I20260811 22:19:26.183248 12883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:26.158905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59220 (local address 127.29.229.62:34795)
0811 22:19:26.159782 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:26.159794 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:26.159838 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:19:26.161499 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:26.161522 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:26.161947 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:26.162283 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:26.164869 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:26.166623 (+ 1754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:26.168532 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:26.169253 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:26.169399 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:26.174992 (+ 5593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:26.175002 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:26.175018 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:26.175065 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:26.178902 (+ 3837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:26.179689 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:26.179703 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:26.179710 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:26.179890 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:26.180286 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:26.180292 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:26.180295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:26.180913 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:26.181280 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:26.181703 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 22:19:26.182160 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":418,"threads_started":1}
I20260811 22:19:26.188823 12877 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 34795 } }
I20260811 22:19:26.189323 12877 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:26.189455 12877 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04f0f45c1bdf4b7991f8f8bbd6d21c59, State: Initialized, Role: FOLLOWER
I20260811 22:19:26.190126 12877 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 34795 } }
I20260811 22:19:26.190352 12877 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:19:26.190459 12877 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:19:26.190582 12877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:26.192880 12877 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 34795 } }
I20260811 22:19:26.193356 12877 leader_election.cc:302] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04f0f45c1bdf4b7991f8f8bbd6d21c59; no voters:
I20260811 22:19:26.194170 12877 leader_election.cc:288] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:19:26.194274 12888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:19:26.195180 12888 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [term 1 LEADER]: Becoming Leader. State: Replica: 04f0f45c1bdf4b7991f8f8bbd6d21c59, State: Running, Role: LEADER
I20260811 22:19:26.195896 12888 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 34795 } }
I20260811 22:19:26.197211 12877 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:19:26.199797 12889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 34795 } } }
I20260811 22:19:26.200045 12893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04f0f45c1bdf4b7991f8f8bbd6d21c59. Latest consensus state: current_term: 1 leader_uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 34795 } } }
I20260811 22:19:26.200254 12889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:26.200320 12893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:26.202973 12904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:19:26.208148 12904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:19:26.217767 12904 catalog_manager.cc:1347] Generated new cluster ID: f2d5e22090f94e8bb4d060b11406ead5
I20260811 22:19:26.217906 12904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:19:26.230911 12904 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:19:26.232666 12904 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:19:26.241348 12904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59: Generated new TSK 0
I20260811 22:19:26.242322 12904 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
I20260811 22:19:26.321555 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:34795
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:26.686682 12921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:26.687036 12921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:26.687208 12921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:26.694895 12921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:26.695086 12921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:26.695266 12921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:19:26.706193 12921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:34795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:26.707630 12921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:26.709113 12921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:26.726673 12930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:26.726764 12934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:26.726712 12931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:26.727620 12921 server_base.cc:1034] running on GCE node
I20260811 22:19:26.728349 12921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:26.729447 12921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:26.730684 12921 hybrid_clock.cc:648] HybridClock initialized: now 1786486766730637 us; error 45 us; skew 500 ppm
I20260811 22:19:26.731077 12921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:26 dist-test-slave-jq4t krb5kdc[12749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486766, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:26.734570 12921 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:19:26.736235 12921 webserver.cc:460] Webserver started at http://127.29.229.1:35739/ using document root <none> and password file <none>
I20260811 22:19:26.737054 12921 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:26.737223 12921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:26.737650 12921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:26.741339 12921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "4c2d9d6d8e3c4bb9be5d6f3ab8b360d6"
format_stamp: "Formatted at 2026-08-11 22:19:26 on dist-test-slave-jq4t"
server_key: "b877a9e5c68c7288c9d5affa9f845c31"
server_key_iv: "9768571d23d0a7ef39b4728988b534c8"
server_key_version: "encryptionkey@0"
I20260811 22:19:26.742254 12921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "4c2d9d6d8e3c4bb9be5d6f3ab8b360d6"
format_stamp: "Formatted at 2026-08-11 22:19:26 on dist-test-slave-jq4t"
server_key: "b877a9e5c68c7288c9d5affa9f845c31"
server_key_iv: "9768571d23d0a7ef39b4728988b534c8"
server_key_version: "encryptionkey@0"
I20260811 22:19:26.748003 12921 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:19:26.751925 12944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:26.753528 12921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:19:26.753676 12921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "4c2d9d6d8e3c4bb9be5d6f3ab8b360d6"
format_stamp: "Formatted at 2026-08-11 22:19:26 on dist-test-slave-jq4t"
server_key: "b877a9e5c68c7288c9d5affa9f845c31"
server_key_iv: "9768571d23d0a7ef39b4728988b534c8"
server_key_version: "encryptionkey@0"
I20260811 22:19:26.753854 12921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:26.773316 12921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:26.778261 12921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:26.778532 12921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:26.779721 12921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:26.781267 12921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:26.781363 12921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:26.781451 12921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:26.781497 12921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:26.820911 12921 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:46561
I20260811 22:19:26.821012 13136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:46561 every 8 connection(s)
I20260811 22:19:26.822273 12921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:19:26.826725 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 12921
I20260811 22:19:26.826970 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:19:26.827898 30612 external_mini_cluster.cc:1428] Setting key 925d83cfeca658a2e3ff85d0b5ae761b
Aug 11 22:19:26 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486766, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:19:26.846186 13143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:26.826182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:49169 (local address 127.29.229.62:34795)
0811 22:19:26.827158 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:26.827164 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:26.828290 (+ 1126us) server_negotiation.cc:407] Connection header received
0811 22:19:26.830637 (+ 2347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:26.830643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:26.830733 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:26.830884 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:26.832650 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:26.833400 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:26.835090 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:26.835513 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:26.835660 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:26.839644 (+ 3984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:26.839651 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:26.839661 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:26.839697 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:26.842418 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:26.843204 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:26.843212 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:26.843217 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:26.843384 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:26.843884 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:26.843891 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:26.843896 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:26.844308 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:26.844513 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:26.845394 (+ 881us) server_negotiation.cc:299] Negotiation successful
0811 22:19:26.845666 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":802,"thread_start_us":586,"threads_started":1}
I20260811 22:19:26.846845 13146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:26.827014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.1:49169)
0811 22:19:26.828082 (+ 1068us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:26.828144 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:26.830336 (+ 2192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:26.831160 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:26.831169 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:26.831776 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:26.832393 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:26.832410 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:26.833607 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:26.833611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:26.834913 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:26.834927 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:26.835786 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:26.835794 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:26.836050 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:26.836883 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:26.836908 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:26.839385 (+ 2477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:26.842588 (+ 3203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:26.842598 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:26.842616 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:26.843064 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:26.843528 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:26.843532 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:26.843535 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:26.843746 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:26.844556 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:26.844569 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:26.844957 (+ 388us) client_negotiation.cc:769] Sending connection context
0811 22:19:26.845222 (+ 265us) client_negotiation.cc:240] Negotiation successful
0811 22:19:26.845654 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":447,"threads_started":1}
I20260811 22:19:26.848722 13139 heartbeater.cc:347] Connected to a master server at 127.29.229.62:34795
I20260811 22:19:26.849103 13139 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:26.850095 13139 heartbeater.cc:511] Master 127.29.229.62:34795 requested a full tablet report, sending...
I20260811 22:19:26.852769 12826 ts_manager.cc:194] Registered new tserver with Master: 4c2d9d6d8e3c4bb9be5d6f3ab8b360d6 (127.29.229.1:46561)
I20260811 22:19:26.854404 12826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:49169
WARNING: no policy specified for kudu/127.29.229.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
I20260811 22:19:26.901439 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:34795
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:27.282018 13150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:27.282409 13150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:27.282599 13150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:27.290514 13150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:27.290719 13150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:27.290949 13150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:19:27.301712 13150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:34795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:27.303251 13150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:27.304716 13150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:27.319882 13157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:27.320192 13158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:27.320796 13160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:27.321383 13150 server_base.cc:1034] running on GCE node
I20260811 22:19:27.322125 13150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:27.323179 13150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:27.324651 13150 hybrid_clock.cc:648] HybridClock initialized: now 1786486767324506 us; error 68 us; skew 500 ppm
I20260811 22:19:27.325084 13150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:27 dist-test-slave-jq4t krb5kdc[12749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486767, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:27.328598 13150 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:19:27.330363 13150 webserver.cc:460] Webserver started at http://127.29.229.2:45901/ using document root <none> and password file <none>
I20260811 22:19:27.331082 13150 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:27.331188 13150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:27.331461 13150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:27.334566 13150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "9e2561875ec04d75a6a9cf97423afb9a"
format_stamp: "Formatted at 2026-08-11 22:19:27 on dist-test-slave-jq4t"
server_key: "ee085a49d4ce3e07946207697d65900e"
server_key_iv: "7f29573a88c1dc69e25bd2a3f8335de1"
server_key_version: "encryptionkey@0"
I20260811 22:19:27.335340 13150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "9e2561875ec04d75a6a9cf97423afb9a"
format_stamp: "Formatted at 2026-08-11 22:19:27 on dist-test-slave-jq4t"
server_key: "ee085a49d4ce3e07946207697d65900e"
server_key_iv: "7f29573a88c1dc69e25bd2a3f8335de1"
server_key_version: "encryptionkey@0"
I20260811 22:19:27.341327 13150 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 22:19:27.345019 13169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:27.346735 13150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:19:27.346928 13150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "9e2561875ec04d75a6a9cf97423afb9a"
format_stamp: "Formatted at 2026-08-11 22:19:27 on dist-test-slave-jq4t"
server_key: "ee085a49d4ce3e07946207697d65900e"
server_key_iv: "7f29573a88c1dc69e25bd2a3f8335de1"
server_key_version: "encryptionkey@0"
I20260811 22:19:27.347118 13150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:27.375346 13150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:27.381192 13150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:27.381565 13150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:27.382999 13150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:27.385814 13150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:27.385917 13150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:27.386061 13150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:27.386126 13150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:27.425000 13150 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:33637
I20260811 22:19:27.425115 13357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:33637 every 8 connection(s)
I20260811 22:19:27.426360 13150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:19:27.430313 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 13150
I20260811 22:19:27.430519 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:19:27.431125 30612 external_mini_cluster.cc:1428] Setting key c4227063fee4142dbe482d43574fba24
Aug 11 22:19:27 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486767, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:19:27.451297 13363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:27.429065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:47167 (local address 127.29.229.62:34795)
0811 22:19:27.429549 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:27.429554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:27.430628 (+ 1074us) server_negotiation.cc:407] Connection header received
0811 22:19:27.433059 (+ 2431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:27.433064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:27.433144 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:27.433251 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:27.436064 (+ 2813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:27.436924 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:27.439547 (+ 2623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:27.440039 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:27.440173 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:27.444900 (+ 4727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:27.444906 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:27.444909 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:27.444946 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:27.447802 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:27.448740 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:27.448746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:27.448749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:27.448894 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:27.449374 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:27.449383 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:27.449385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:27.449692 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:27.449865 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:27.450652 (+ 787us) server_negotiation.cc:299] Negotiation successful
0811 22:19:27.450920 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":281,"threads_started":1}
I20260811 22:19:27.452544 13364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:27.429487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.2:47167)
0811 22:19:27.430386 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:27.430449 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:27.432773 (+ 2324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:27.433926 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:27.433943 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:27.434928 (+ 985us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:27.435811 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:27.435845 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:27.437123 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:27.437133 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:27.439364 (+ 2231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:27.439384 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:27.440313 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:27.440324 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:27.440686 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:27.441518 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:27.441553 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:27.444630 (+ 3077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:27.447966 (+ 3336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:27.447976 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:27.448003 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:27.448608 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:27.449025 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:27.449030 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:27.449034 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:27.449242 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:27.449845 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:27.449853 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:27.450246 (+ 393us) client_negotiation.cc:769] Sending connection context
0811 22:19:27.450552 (+ 306us) client_negotiation.cc:240] Negotiation successful
0811 22:19:27.451235 (+ 683us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":336,"threads_started":1}
I20260811 22:19:27.454236 13359 heartbeater.cc:347] Connected to a master server at 127.29.229.62:34795
I20260811 22:19:27.454631 13359 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:27.455544 13359 heartbeater.cc:511] Master 127.29.229.62:34795 requested a full tablet report, sending...
I20260811 22:19:27.457536 12826 ts_manager.cc:194] Registered new tserver with Master: 9e2561875ec04d75a6a9cf97423afb9a (127.29.229.2:33637)
I20260811 22:19:27.458873 12826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:47167
WARNING: no policy specified for kudu/127.29.229.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
I20260811 22:19:27.507587 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:34795
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:19:27.858703 13139 heartbeater.cc:503] Master 127.29.229.62:34795 was elected leader, sending a full tablet report...
W20260811 22:19:27.882046 13367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:27.882429 13367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:27.882616 13367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:27.890067 13367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:27.890357 13367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:27.890574 13367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:19:27.900864 13367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:34795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:27.902383 13367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:27.903893 13367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:27.920254 13379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:27.920506 13381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:27.920500 13384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:27.922293 13367 server_base.cc:1034] running on GCE node
I20260811 22:19:27.923126 13367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:27.924216 13367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:27.925508 13367 hybrid_clock.cc:648] HybridClock initialized: now 1786486767925456 us; error 37 us; skew 500 ppm
I20260811 22:19:27.925900 13367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:27 dist-test-slave-jq4t krb5kdc[12749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486767, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:27.929774 13367 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:19:27.931510 13367 webserver.cc:460] Webserver started at http://127.29.229.3:33035/ using document root <none> and password file <none>
I20260811 22:19:27.932262 13367 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:27.932425 13367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:27.932799 13367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:27.935886 13367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "5cf845d5e678464ba2cbe169afa67939"
format_stamp: "Formatted at 2026-08-11 22:19:27 on dist-test-slave-jq4t"
server_key: "0b2dd87b037b367f275e0ef93c1eab2c"
server_key_iv: "867269133366f25f3f2f349ffdae4fda"
server_key_version: "encryptionkey@0"
I20260811 22:19:27.936942 13367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "5cf845d5e678464ba2cbe169afa67939"
format_stamp: "Formatted at 2026-08-11 22:19:27 on dist-test-slave-jq4t"
server_key: "0b2dd87b037b367f275e0ef93c1eab2c"
server_key_iv: "867269133366f25f3f2f349ffdae4fda"
server_key_version: "encryptionkey@0"
I20260811 22:19:27.942675 13367 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 22:19:27.946825 13395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:27.948298 13367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:19:27.948568 13367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "5cf845d5e678464ba2cbe169afa67939"
format_stamp: "Formatted at 2026-08-11 22:19:27 on dist-test-slave-jq4t"
server_key: "0b2dd87b037b367f275e0ef93c1eab2c"
server_key_iv: "867269133366f25f3f2f349ffdae4fda"
server_key_version: "encryptionkey@0"
I20260811 22:19:27.948793 13367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:27.975064 13367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:27.980126 13367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:27.980566 13367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:27.981921 13367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:27.983467 13367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:27.983584 13367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:27.983745 13367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:27.983822 13367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:28.022678 13367 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:46151
I20260811 22:19:28.022717 13603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:46151 every 8 connection(s)
I20260811 22:19:28.024017 13367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:19:28.032835 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 13367
I20260811 22:19:28.033028 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:19:28.033552 30612 external_mini_cluster.cc:1428] Setting key 2107f25129511c550d7424d316348106
Aug 11 22:19:28 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486767, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:19:28.048774 13607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:28.027630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:60711 (local address 127.29.229.62:34795)
0811 22:19:28.028162 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:28.028167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:28.029242 (+ 1075us) server_negotiation.cc:407] Connection header received
0811 22:19:28.031245 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:28.031251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:28.031351 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:28.031481 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:28.033280 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:28.034205 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:28.036376 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:28.037037 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:28.037142 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:28.041097 (+ 3955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:28.041102 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:28.041105 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:28.041291 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:28.044721 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:28.045705 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:28.045710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:28.045713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:28.045910 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:28.046475 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:28.046479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:28.046481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:28.046791 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:28.046968 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:28.048004 (+ 1036us) server_negotiation.cc:299] Negotiation successful
0811 22:19:28.048273 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":276,"threads_started":1}
I20260811 22:19:28.050043 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:28.028247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.3:60711)
0811 22:19:28.029069 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:28.029111 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:28.030952 (+ 1841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:28.031735 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:28.031744 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:28.032299 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:28.033083 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:28.033115 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:28.034408 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:28.034414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:28.036217 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:28.036231 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:28.037368 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:28.037374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:28.037598 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:28.038313 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:28.038338 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:28.040858 (+ 2520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:28.044957 (+ 4099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:28.044965 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:28.044979 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:28.045498 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:28.046068 (+ 570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:28.046072 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:28.046075 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:28.046320 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:28.046954 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:28.046961 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:28.047429 (+ 468us) client_negotiation.cc:769] Sending connection context
0811 22:19:28.047793 (+ 364us) client_negotiation.cc:240] Negotiation successful
0811 22:19:28.048403 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":356,"threads_started":1}
I20260811 22:19:28.051894 13604 heartbeater.cc:347] Connected to a master server at 127.29.229.62:34795
I20260811 22:19:28.052268 13604 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:28.053194 13604 heartbeater.cc:511] Master 127.29.229.62:34795 requested a full tablet report, sending...
I20260811 22:19:28.054911 12826 ts_manager.cc:194] Registered new tserver with Master: 5cf845d5e678464ba2cbe169afa67939 (127.29.229.3:46151)
I20260811 22:19:28.055773 12826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:60711
I20260811 22:19:28.063051 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.1.keytab.
Entry for principal oryx/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.1.keytab.
WARNING: no policy specified for oryx/127.29.229.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.229.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.2.keytab.
Entry for principal oryx/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.2.keytab.
WARNING: no policy specified for oryx/127.29.229.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.3.keytab.
Entry for principal oryx/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.3.keytab.
I20260811 22:19:28.273262 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 12921
Aug 11 22:19:28 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:19:28.290241 13621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:28.274483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57332 (local address 127.29.229.1:46561)
0811 22:19:28.275165 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:28.275172 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:28.275200 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:19:28.275266 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:28.275272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:28.275495 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:28.275636 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:28.277085 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:28.278129 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:28.279351 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:28.279936 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:28.280053 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:28.284315 (+ 4262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:28.284327 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:28.284344 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:28.284386 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:28.287489 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:28.288246 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:28.288253 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:28.288255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:28.288389 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:28.288810 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:28.288813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:28.288815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:28.289180 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:28.289417 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:28.289717 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 22:19:28.289932 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":354,"threads_started":1}
I20260811 22:19:28.291579 13131 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:28.378738 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 12921
I20260811 22:19:28.406919 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 13150
Aug 11 22:19:28 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.2@KRBTEST.COM
I20260811 22:19:28.423418 13624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:28.407833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38672 (local address 127.29.229.2:33637)
0811 22:19:28.408391 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:28.408398 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:28.408432 (+ 34us) server_negotiation.cc:407] Connection header received
0811 22:19:28.408582 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:28.408589 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:28.408805 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:28.408938 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:28.410214 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:28.411242 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:28.412621 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:28.413181 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:28.413284 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:28.417558 (+ 4274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:28.417570 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:28.417584 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:28.417628 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:28.420625 (+ 2997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:28.421328 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:28.421336 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:28.421339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:28.421495 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:28.421827 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:28.421831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:28.421833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:28.422210 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:28.422454 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:28.422771 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 22:19:28.422994 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":294,"threads_started":1}
I20260811 22:19:28.424911 13353 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:28.511791 13359 heartbeater.cc:503] Master 127.29.229.62:34795 was elected leader, sending a full tablet report...
I20260811 22:19:28.517588 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 13150
W20260811 22:19:28.539166 12799 connection.cc:535] server connection from 127.29.229.2:47167 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:19:28.539419 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 13367
Aug 11 22:19:28 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.3@KRBTEST.COM
I20260811 22:19:28.558509 13636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:28.540552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46688 (local address 127.29.229.3:46151)
0811 22:19:28.541141 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:28.541149 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:28.541181 (+ 32us) server_negotiation.cc:407] Connection header received
0811 22:19:28.541288 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:28.541297 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:28.541570 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:28.541709 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:28.543068 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:28.544129 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:28.545612 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:28.546197 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:28.546316 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:28.551144 (+ 4828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:28.551160 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:28.551175 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:28.551213 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:28.554815 (+ 3602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:28.555600 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:28.555607 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:28.555609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:28.555737 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:28.556153 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:28.556158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:28.556160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:28.556632 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:28.557266 (+ 634us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:28.557795 (+ 529us) server_negotiation.cc:299] Negotiation successful
0811 22:19:28.558137 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":260,"threads_started":1}
I20260811 22:19:28.559918 13596 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:28.645956 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 13367
I20260811 22:19:28.675565 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 12765
I20260811 22:19:28.676997 12871 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:28.787119 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 12765
I20260811 22:19:28.819844 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:34795
--webserver_interface=127.29.229.62
--webserver_port=42069
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:34795
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:29.191517 13639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:29.191844 13639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:29.191939 13639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:29.199329 13639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:19:29.199447 13639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:29.199496 13639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:29.199546 13639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:19:29.199594 13639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:19:29.210000 13639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:34795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:34795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=42069
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:29.211364 13639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:29.212749 13639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:29.225030 13645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:29.225163 13646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:29.225142 13648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:29.226313 13639 server_base.cc:1034] running on GCE node
I20260811 22:19:29.227206 13639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:29.228709 13639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:29.229930 13639 hybrid_clock.cc:648] HybridClock initialized: now 1786486769229875 us; error 45 us; skew 500 ppm
I20260811 22:19:29.230252 13639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:29 dist-test-slave-jq4t krb5kdc[12749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486769, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:29.237658 13639 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:19:29.239310 13639 webserver.cc:460] Webserver started at http://127.29.229.62:42069/ using document root <none> and password file <none>
I20260811 22:19:29.240065 13639 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:29.240236 13639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:29.246042 13639 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:19:29.249902 13660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:29.251616 13639 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 22:19:29.251861 13639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59"
format_stamp: "Formatted at 2026-08-11 22:19:26 on dist-test-slave-jq4t"
server_key: "17c68564a144f6cd53409f6c0fb1be90"
server_key_iv: "114bba7eeeb39f0c6f523feca2d5d46c"
server_key_version: "encryptionkey@0"
I20260811 22:19:29.253037 13639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:29.276396 13639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:29.281592 13639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:29.281886 13639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:29.303359 13639 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:34795
I20260811 22:19:29.303412 13742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:34795 every 8 connection(s)
I20260811 22:19:29.305135 13639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:19:29.312376 13745 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:19:29.313661 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 13639
I20260811 22:19:29.314714 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:46561
--local_ip_for_outbound_sockets=127.29.229.1
--tserver_master_addrs=127.29.229.62:34795
--webserver_port=35739
--webserver_interface=127.29.229.1
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.229.1
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:19:29.317498 13745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59: Bootstrap starting.
I20260811 22:19:29.343405 13745 log.cc:824] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:29.354077 13745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:19:29.354660 13745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59: Bootstrap complete.
I20260811 22:19:29.364336 13745 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 34795 } }
I20260811 22:19:29.365288 13745 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04f0f45c1bdf4b7991f8f8bbd6d21c59, State: Initialized, Role: FOLLOWER
I20260811 22:19:29.366103 13745 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 34795 } }
I20260811 22:19:29.366340 13745 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:19:29.366475 13745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:19:29.366680 13745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:19:29.370965 13745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 34795 } }
I20260811 22:19:29.371518 13745 leader_election.cc:302] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04f0f45c1bdf4b7991f8f8bbd6d21c59; no voters:
I20260811 22:19:29.372200 13745 leader_election.cc:288] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:19:29.372354 13749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:19:29.373687 13749 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [term 2 LEADER]: Becoming Leader. State: Replica: 04f0f45c1bdf4b7991f8f8bbd6d21c59, State: Running, Role: LEADER
I20260811 22:19:29.374276 13749 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 34795 } }
I20260811 22:19:29.374796 13745 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:19:29.377125 13751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04f0f45c1bdf4b7991f8f8bbd6d21c59. Latest consensus state: current_term: 2 leader_uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 34795 } } }
I20260811 22:19:29.377599 13751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:29.378316 13750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f0f45c1bdf4b7991f8f8bbd6d21c59" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 34795 } } }
I20260811 22:19:29.378736 13750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:29.404670 13761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:19:29.410457 13761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:19:29.411935 13761 catalog_manager.cc:1259] Loaded cluster ID: f2d5e22090f94e8bb4d060b11406ead5
I20260811 22:19:29.412045 13761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:19:29.416087 13761 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:19:29.417572 13761 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 04f0f45c1bdf4b7991f8f8bbd6d21c59: Loaded TSK: 0
I20260811 22:19:29.419178 13761 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:19:29.697233 13747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:29.697592 13747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:29.697762 13747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:29.705415 13747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:29.705593 13747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:29.705780 13747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:19:29.716045 13747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.1.keytab
--principal=oryx/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:46561
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=35739
--tserver_master_addrs=127.29.229.62:34795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:29.717952 13747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:29.719415 13747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:29.735781 13788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:29.736019 13795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:29.736848 13791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:29.737586 13747 server_base.cc:1034] running on GCE node
I20260811 22:19:29.738859 13747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:29.740880 13747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:29.742213 13747 hybrid_clock.cc:648] HybridClock initialized: now 1786486769742092 us; error 86 us; skew 500 ppm
I20260811 22:19:29.742739 13747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:29 dist-test-slave-jq4t krb5kdc[12749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486769, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:29.748121 13747 init.cc:377] Logged in from keytab as oryx/127.29.229.1@KRBTEST.COM (short username oryx)
I20260811 22:19:29.750607 13747 webserver.cc:460] Webserver started at http://127.29.229.1:35739/ using document root <none> and password file <none>
I20260811 22:19:29.751665 13747 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:29.751850 13747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:29.760427 13747 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 22:19:29.764310 13806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:29.766050 13747 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:19:29.766239 13747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "4c2d9d6d8e3c4bb9be5d6f3ab8b360d6"
format_stamp: "Formatted at 2026-08-11 22:19:26 on dist-test-slave-jq4t"
server_key: "b877a9e5c68c7288c9d5affa9f845c31"
server_key_iv: "9768571d23d0a7ef39b4728988b534c8"
server_key_version: "encryptionkey@0"
I20260811 22:19:29.767608 13747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:29.793730 13747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:29.799504 13747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:29.799820 13747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:29.801108 13747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:29.803335 13747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:29.803457 13747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:29.803576 13747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:29.803651 13747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:29.844568 13747 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:46561
I20260811 22:19:29.844652 13995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:46561 every 8 connection(s)
I20260811 22:19:29.846298 13747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:19:29.853101 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 13747
I20260811 22:19:29.853734 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:33637
--local_ip_for_outbound_sockets=127.29.229.2
--tserver_master_addrs=127.29.229.62:34795
--webserver_port=45901
--webserver_interface=127.29.229.2
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.229.2
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:19:29 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:29 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:19:30.240620 14000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:30.240912 14000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:30.241066 14000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:30.248594 14000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:30.248725 14000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:30.248895 14000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:19:30.259675 14000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.2.keytab
--principal=oryx/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:33637
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=45901
--tserver_master_addrs=127.29.229.62:34795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:30.262022 14000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:30.263484 14000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:30.282915 14009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:30.283013 14012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:30.283668 14008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:30.284847 14000 server_base.cc:1034] running on GCE node
I20260811 22:19:30.286065 14000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:30.287901 14000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:30.289228 14000 hybrid_clock.cc:648] HybridClock initialized: now 1786486770289138 us; error 54 us; skew 500 ppm
I20260811 22:19:30.289819 14000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:30 dist-test-slave-jq4t krb5kdc[12749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486770, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:30.295364 14000 init.cc:377] Logged in from keytab as oryx/127.29.229.2@KRBTEST.COM (short username oryx)
I20260811 22:19:30.297237 14000 webserver.cc:460] Webserver started at http://127.29.229.2:45901/ using document root <none> and password file <none>
I20260811 22:19:30.297978 14000 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:30.298148 14000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:30.305425 14000 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260811 22:19:30.309356 14021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:30.311259 14000 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260811 22:19:30.311455 14000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "9e2561875ec04d75a6a9cf97423afb9a"
format_stamp: "Formatted at 2026-08-11 22:19:27 on dist-test-slave-jq4t"
server_key: "ee085a49d4ce3e07946207697d65900e"
server_key_iv: "7f29573a88c1dc69e25bd2a3f8335de1"
server_key_version: "encryptionkey@0"
I20260811 22:19:30.312836 14000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:30.325255 14000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:30.331070 14000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:30.331410 14000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:30.332717 14000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:30.334292 14000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:30.334393 14000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:30.334547 14000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:30.334625 14000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:30.383692 14000 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:33637
I20260811 22:19:30.383762 14194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:33637 every 8 connection(s)
I20260811 22:19:30.385277 14000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:19:30.394253 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 14000
I20260811 22:19:30.395364 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:46151
--local_ip_for_outbound_sockets=127.29.229.3
--tserver_master_addrs=127.29.229.62:34795
--webserver_port=33035
--webserver_interface=127.29.229.3
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.229.3
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:19:30 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:30 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:19:30.784384 14202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:30.784782 14202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:30.784965 14202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:30.793484 14202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:30.793639 14202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:30.793830 14202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:19:30.804306 14202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/krb5kdc/oryx_127.29.229.3.keytab
--principal=oryx/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:46151
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=33035
--tserver_master_addrs=127.29.229.62:34795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:30.805748 14202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:30.807075 14202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:30.826476 14215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:30.831296 14214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:30.831931 14217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:30.833443 14202 server_base.cc:1034] running on GCE node
I20260811 22:19:30.834216 14202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:30.835332 14202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:30.836648 14202 hybrid_clock.cc:648] HybridClock initialized: now 1786486770836564 us; error 59 us; skew 500 ppm
I20260811 22:19:30.837028 14202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:30 dist-test-slave-jq4t krb5kdc[12749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486770, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:30.840603 14202 init.cc:377] Logged in from keytab as oryx/127.29.229.3@KRBTEST.COM (short username oryx)
I20260811 22:19:30.858543 14202 webserver.cc:460] Webserver started at http://127.29.229.3:33035/ using document root <none> and password file <none>
I20260811 22:19:30.859396 14202 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:30.859586 14202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:30.865674 14202 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:19:30.868682 14231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:30.870239 14202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:19:30.870464 14202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "5cf845d5e678464ba2cbe169afa67939"
format_stamp: "Formatted at 2026-08-11 22:19:27 on dist-test-slave-jq4t"
server_key: "0b2dd87b037b367f275e0ef93c1eab2c"
server_key_iv: "867269133366f25f3f2f349ffdae4fda"
server_key_version: "encryptionkey@0"
I20260811 22:19:30.871608 14202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:30.889231 14202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:30.894089 14202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:30.894420 14202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:30.895581 14202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:30.897217 14202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:30.897365 14202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:30.897512 14202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:30.897604 14202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:30.948796 14202 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:46151
I20260811 22:19:30.948840 14407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:46151 every 8 connection(s)
I20260811 22:19:30.950484 14202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:19:30.952863 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 14202
Aug 11 22:19:30 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:30 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:30.989394 14413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:30.954282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59224 (local address 127.29.229.62:34795)
0811 22:19:30.964760 (+ 10478us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:30.964768 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:30.964801 (+ 33us) server_negotiation.cc:407] Connection header received
0811 22:19:30.964890 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:30.964895 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:30.964991 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:30.968782 (+ 3791us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:30.969108 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:30.970329 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:30.972297 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:30.972929 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:30.973064 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:30.977493 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:30.977508 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:30.977534 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:30.977589 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:30.982446 (+ 4857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:30.983604 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:30.983616 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:30.983629 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:30.983849 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:30.984333 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:30.984341 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:30.984347 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:30.985073 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:30.986786 (+ 1713us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:30.987096 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 22:19:30.987614 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10318,"thread_start_us":10520,"threads_started":1}
W20260811 22:19:31.407433 13999 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.547s user 0.000s sys 0.019s
W20260811 22:19:31.409745 13998 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:29.849611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:40855 (local address 127.29.229.62:34795)
0811 22:19:29.850513 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:29.850530 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:29.850765 (+ 235us) server_negotiation.cc:407] Connection header received
0811 22:19:29.853029 (+ 2264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:29.853041 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:29.853409 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:29.853748 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:29.855284 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:29.856590 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:29.858235 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:29.858867 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:29.859066 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:31.409234 (+1550168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.1:40855: BlockingRecv error: recv got EOF from 127.29.229.1:40855 (error 108)
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":360,"threads_started":1}
I20260811 22:19:31.410687 13999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:29.849680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.1:40855)
0811 22:19:29.850573 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:29.850631 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:29.852391 (+ 1760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:29.853916 (+ 1525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:29.853926 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:29.854548 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:29.855066 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:29.855082 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:29.856750 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:29.856755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:29.858081 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:29.858094 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:29.859154 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:29.859163 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:29.859434 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:29.860433 (+ 999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:29.860653 (+ 220us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:31.408774 (+1548121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":380,"threads_started":1}
W20260811 22:19:31.412371 13996 heartbeater.cc:644] Failed to heartbeat to 127.29.229.62:34795 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.229.62:34795: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Aug 11 22:19:31 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:31 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:19:31.431416 13998 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:31.413695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:35797 (local address 127.29.229.62:34795)
0811 22:19:31.413965 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:31.413975 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:31.414007 (+ 32us) server_negotiation.cc:407] Connection header received
0811 22:19:31.414292 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:31.414297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:31.414419 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:31.414579 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:31.416030 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:31.418034 (+ 2004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:31.421041 (+ 3007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:31.422066 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:31.422263 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:31.431010 (+ 8747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.1:35797: BlockingRecv error: recv got EOF from 127.29.229.1:35797 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 22:19:31.432579 13999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:31.413485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.1:35797)
0811 22:19:31.413904 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:31.413935 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:31.414104 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:31.414666 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:31.414674 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:31.415401 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:31.415774 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:31.415792 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:31.418367 (+ 2575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:31.418377 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:31.420817 (+ 2440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:31.420841 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:31.422378 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:31.422388 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:31.422685 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:31.423758 (+ 1073us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:31.423796 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:31.432122 (+ 8326us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234}
Aug 11 22:19:31 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:31 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:31.450515 13999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:31.433748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.1:46353)
0811 22:19:31.434115 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:31.434142 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:31.434307 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:31.434872 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:31.434880 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:31.435506 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:31.435847 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:31.435865 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:31.438079 (+ 2214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:31.438088 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:31.440429 (+ 2341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:31.440453 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:31.442292 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:31.442298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:31.442597 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:31.443608 (+ 1011us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:31.443646 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:31.450057 (+ 6411us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199}
W20260811 22:19:31.451114 13998 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:31.433881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:46353 (local address 127.29.229.62:34795)
0811 22:19:31.434153 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:31.434162 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:31.434256 (+ 94us) server_negotiation.cc:407] Connection header received
0811 22:19:31.434473 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:31.434482 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:31.434628 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:31.434781 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:31.436094 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:31.437863 (+ 1769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:31.440710 (+ 2847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:31.441699 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:31.441855 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:31.450684 (+ 8829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.1:46353: BlockingRecv error: recv got EOF from 127.29.229.1:46353 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
W20260811 22:19:31.451290 13996 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:19:31.880376 14201 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.480s user 0.004s sys 0.015s
W20260811 22:19:31.881803 14199 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:30.388331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:45215 (local address 127.29.229.62:34795)
0811 22:19:30.389126 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:30.389136 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:30.389989 (+ 853us) server_negotiation.cc:407] Connection header received
0811 22:19:30.392072 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:30.392082 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:30.392237 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:30.392433 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:30.394475 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:30.395329 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:30.397929 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:30.398580 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:30.398796 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:31.881464 (+1482668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.2:45215: BlockingRecv error: recv got EOF from 127.29.229.2:45215 (error 108)
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":364,"threads_started":1}
I20260811 22:19:31.882863 14201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:30.388891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.2:45215)
0811 22:19:30.389779 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:30.389829 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:30.391739 (+ 1910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:30.392764 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:30.392774 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:30.393376 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:30.394224 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:30.394247 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:30.395548 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:30.395555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:30.397400 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:30.397432 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:30.398992 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:30.399003 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:30.399355 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:30.400373 (+ 1018us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:30.400404 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:31.881256 (+1480852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":376,"threads_started":1}
W20260811 22:19:31.884076 14195 heartbeater.cc:644] Failed to heartbeat to 127.29.229.62:34795 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.229.62:34795: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Aug 11 22:19:31 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:31 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:31.897099 14201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:31.885023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.2:36175)
0811 22:19:31.885344 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:31.885373 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:31.885509 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:31.885958 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:31.885962 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:31.886338 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:31.886527 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:31.886536 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:31.888690 (+ 2154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:31.888695 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:31.889841 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:31.889855 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:31.891053 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:31.891060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:31.891253 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:31.892154 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:31.892174 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:31.896751 (+ 4577us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197}
W20260811 22:19:31.897207 14199 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:31.885012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:36175 (local address 127.29.229.62:34795)
0811 22:19:31.885183 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:31.885188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:31.885479 (+ 291us) server_negotiation.cc:407] Connection header received
0811 22:19:31.885630 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:31.885638 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:31.885759 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:31.885912 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:31.886700 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:31.888449 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:31.890153 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:31.890833 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:31.890998 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:31.896972 (+ 5974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.2:36175: BlockingRecv error: recv got EOF from 127.29.229.2:36175 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 22:19:31 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:31 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:31.909714 14201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:31.898039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.2:45883)
0811 22:19:31.898264 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:31.898286 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:31.898393 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:31.899039 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:31.899045 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:31.899557 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:31.899848 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:31.899860 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:31.901951 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:31.901955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:31.903353 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:31.903380 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:31.904326 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:31.904331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:31.904571 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:31.905391 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:31.905410 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:31.909412 (+ 4002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95}
W20260811 22:19:31.909924 14199 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:31.898220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:45883 (local address 127.29.229.62:34795)
0811 22:19:31.898464 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:31.898474 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:31.898503 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:19:31.898584 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:31.898594 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:31.898725 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:31.898877 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:31.900018 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:31.901734 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:31.903561 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:31.904149 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:31.904306 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:31.909628 (+ 5322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.2:45883: BlockingRecv error: recv got EOF from 127.29.229.2:45883 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 22:19:31.910190 14195 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:19:32.223690 14412 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.248s user 0.008s sys 0.015s
W20260811 22:19:32.225438 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:30.958853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:55899 (local address 127.29.229.62:34795)
0811 22:19:30.965029 (+ 6176us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:30.965036 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:30.965055 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:19:30.965098 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:30.965117 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:30.965340 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:30.965514 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:30.968878 (+ 3364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:30.970330 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:30.972935 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:30.973669 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:30.973781 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:32.225037 (+1251256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.3:55899: BlockingRecv error: recv got EOF from 127.29.229.3:55899 (error 108)
Metrics: {"server-negotiator.queue_time_us":6068,"thread_start_us":5619,"threads_started":1}
I20260811 22:19:32.225615 14412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:30.958105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.3:55899)
0811 22:19:30.959140 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:30.959193 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:30.962339 (+ 3146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:30.966828 (+ 4489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:30.966844 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:30.967707 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:30.968656 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:30.968689 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:30.970569 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:30.970575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:30.972769 (+ 2194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:30.972785 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:30.974246 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:30.974258 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:30.974699 (+ 441us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:30.975532 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:30.975567 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:32.224629 (+1249062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":728,"threads_started":1}
W20260811 22:19:32.226606 14408 heartbeater.cc:644] Failed to heartbeat to 127.29.229.62:34795 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.229.62:34795: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Aug 11 22:19:32 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:32 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:19:32.238201 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:32.227503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:48909 (local address 127.29.229.62:34795)
0811 22:19:32.227659 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:32.227664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:32.227683 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:19:32.227848 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:32.227852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:32.227923 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:32.228005 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:32.228872 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:32.229627 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:32.230954 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:32.231577 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:32.231695 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:32.238005 (+ 6310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.3:48909: BlockingRecv error: recv got EOF from 127.29.229.3:48909 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:19:32.238250 14412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:32.227369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.3:48909)
0811 22:19:32.227596 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:32.227622 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:32.227733 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:32.228069 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:32.228073 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:32.228441 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:32.228693 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:32.228703 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:32.229814 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:32.229818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:32.230771 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:32.230783 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:32.231781 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:32.231786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:32.231927 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:32.232798 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:32.232939 (+ 141us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:32.237768 (+ 4829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
Aug 11 22:19:32 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:32 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:32.248425 14412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:32.239035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.3:59117)
0811 22:19:32.239227 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:32.239244 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:32.239400 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:32.239801 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:32.239806 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:32.240184 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:32.240352 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:32.240370 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:32.241448 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:32.241453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:32.242542 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:32.242560 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:32.243446 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:32.243452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:32.243664 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:32.244313 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:32.244331 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:32.248092 (+ 3761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 22:19:32.248634 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:32.239178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:59117 (local address 127.29.229.62:34795)
0811 22:19:32.239489 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:32.239495 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:32.239513 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:19:32.239571 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:32.239575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:32.239646 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:32.239757 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:32.240530 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:32.241291 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:32.242696 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:32.243287 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:32.243424 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:32.248313 (+ 4889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.3:59117: BlockingRecv error: recv got EOF from 127.29.229.3:59117 (error 108)
Metrics: {"server-negotiator.queue_time_us":186}
W20260811 22:19:32.248899 14408 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 22:19:32 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:32 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:32.462612 14415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:32.451962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.1:47025)
0811 22:19:32.452853 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:32.452876 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:32.453052 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:32.453463 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:32.453467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:32.453980 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:32.454134 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:32.454143 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:32.455185 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:32.455191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:32.456312 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:32.456327 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:32.457367 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:32.457374 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:32.457556 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:32.458278 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:32.458298 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:32.462339 (+ 4041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":783,"thread_start_us":529,"threads_started":1}
W20260811 22:19:32.462814 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:32.452069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:47025 (local address 127.29.229.62:34795)
0811 22:19:32.452652 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:32.452660 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:32.452977 (+ 317us) server_negotiation.cc:407] Connection header received
0811 22:19:32.453205 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:32.453209 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:32.453302 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:32.453436 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:32.454295 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:32.455024 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:32.456567 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:32.457162 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:32.457326 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:32.462565 (+ 5239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.1:47025: BlockingRecv error: recv got EOF from 127.29.229.1:47025 (error 108)
Metrics: {"server-negotiator.queue_time_us":200}
Aug 11 22:19:32 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:32 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:32.920909 14417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:32.910863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.2:53183)
0811 22:19:32.911456 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:32.911474 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:32.911602 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:32.911980 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:32.911983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:32.912429 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:32.912624 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:32.912633 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:32.913792 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:32.913799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:32.915247 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:32.915263 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:32.916068 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:32.916073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:32.916193 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:32.916935 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:32.916953 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:32.920645 (+ 3692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":358,"threads_started":1}
W20260811 22:19:32.921113 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:32.911023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:53183 (local address 127.29.229.62:34795)
0811 22:19:32.911228 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:32.911232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:32.911567 (+ 335us) server_negotiation.cc:407] Connection header received
0811 22:19:32.911750 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:32.911754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:32.911826 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:32.911934 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:32.912786 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:32.913602 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:32.915406 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:32.915890 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:32.915995 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:32.920854 (+ 4859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.2:53183: BlockingRecv error: recv got EOF from 127.29.229.2:53183 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 22:19:33 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:33 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:33.259797 14418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:33.249562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.3:60069)
0811 22:19:33.250143 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:33.250159 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:33.250332 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:33.250735 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:33.250739 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:33.251258 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:33.251416 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:33.251426 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:33.252552 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:33.252556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:33.253720 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:33.253734 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:33.254520 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:33.254528 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:33.254715 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:33.255354 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:33.255372 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:33.259430 (+ 4058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":344,"threads_started":1}
W20260811 22:19:33.259864 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:33.249638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:60069 (local address 127.29.229.62:34795)
0811 22:19:33.249924 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:33.249932 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:33.250259 (+ 327us) server_negotiation.cc:407] Connection header received
0811 22:19:33.250488 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:33.250495 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:33.250580 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:33.250695 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:33.251577 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:33.252360 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:33.253870 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:33.254325 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:33.254482 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:33.259647 (+ 5165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.3:60069: BlockingRecv error: recv got EOF from 127.29.229.3:60069 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 11 22:19:33 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:33 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:33.474396 14419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:33.463772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.1:52459)
0811 22:19:33.464318 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:33.464333 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:33.464464 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:33.464814 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:33.464817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:33.465222 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:33.465400 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:33.465408 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:33.466513 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:33.466518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:33.468060 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:33.468100 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:33.468993 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:33.468997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:33.469131 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:33.469801 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:33.469820 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:33.474101 (+ 4281us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":319,"threads_started":1}
W20260811 22:19:33.474929 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:33.463868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:52459 (local address 127.29.229.62:34795)
0811 22:19:33.464154 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:33.464161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:33.464425 (+ 264us) server_negotiation.cc:407] Connection header received
0811 22:19:33.464594 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:33.464598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:33.464664 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:33.464750 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:33.465571 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:33.466349 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:33.468283 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:33.468808 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:33.468937 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:33.474572 (+ 5635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.1:52459: BlockingRecv error: recv got EOF from 127.29.229.1:52459 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 11 22:19:33 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:33 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:33.931843 14421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:33.921975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.2:51895)
0811 22:19:33.922539 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:33.922556 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:33.922674 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:33.922992 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:33.922995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:33.923379 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:33.923567 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:33.923575 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:33.924638 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:33.924642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:33.925565 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:33.925576 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:33.926315 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:33.926319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:33.926449 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:33.927228 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:33.927246 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:33.931466 (+ 4220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":319,"threads_started":1}
W20260811 22:19:33.931882 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:33.922036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:51895 (local address 127.29.229.62:34795)
0811 22:19:33.922244 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:33.922248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:33.922636 (+ 388us) server_negotiation.cc:407] Connection header received
0811 22:19:33.922774 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:33.922778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:33.922846 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:33.922942 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:33.923705 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:33.924469 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:33.925717 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:33.926171 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:33.926269 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:33.931650 (+ 5381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.2:51895: BlockingRecv error: recv got EOF from 127.29.229.2:51895 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 22:19:34 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:34 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:34.274463 14422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:34.260966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.3:52093)
0811 22:19:34.261548 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:34.261564 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:34.261682 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:34.262030 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:34.262033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:34.262437 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:34.262630 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:34.262638 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:34.263709 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:34.263713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:34.264681 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:34.264693 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:34.265388 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:34.265393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:34.265522 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:34.270181 (+ 4659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:34.270200 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:34.274090 (+ 3890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":340,"threads_started":1}
W20260811 22:19:34.274521 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:34.261092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:52093 (local address 127.29.229.62:34795)
0811 22:19:34.261289 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:34.261294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:34.261645 (+ 351us) server_negotiation.cc:407] Connection header received
0811 22:19:34.261792 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:34.261796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:34.261869 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:34.261958 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:34.262788 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:34.263581 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:34.264825 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:34.265249 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:34.265355 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:34.274290 (+ 8935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.3:52093: BlockingRecv error: recv got EOF from 127.29.229.3:52093 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 11 22:19:34 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:34 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:34.485267 14423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:34.475617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.1:46053)
0811 22:19:34.476192 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:34.476209 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:34.476331 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:34.476632 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:34.476635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:34.477078 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:34.477259 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:34.477268 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:34.478396 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:34.478401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:34.479624 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:34.479637 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:34.480606 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:34.480611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:34.480827 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:34.481468 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:34.481486 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:34.485019 (+ 3533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":307,"threads_started":1}
W20260811 22:19:34.485646 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:34.475684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:46053 (local address 127.29.229.62:34795)
0811 22:19:34.475938 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:34.475949 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:34.476300 (+ 351us) server_negotiation.cc:407] Connection header received
0811 22:19:34.476434 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:34.476438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:34.476522 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:34.476602 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:34.477396 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:34.478266 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:34.479788 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:34.480385 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:34.480545 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:34.485240 (+ 4695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.1:46053: BlockingRecv error: recv got EOF from 127.29.229.1:46053 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 22:19:34 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:34 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:34.943001 14424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:34.933188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.2:58653)
0811 22:19:34.933763 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:34.933788 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:34.933924 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:34.934437 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:34.934441 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:34.934841 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:34.935029 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:34.935039 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:34.936354 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:34.936360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:34.937385 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:34.937398 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:34.938164 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:34.938170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:34.938375 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:34.939054 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:34.939072 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:34.942744 (+ 3672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":317,"threads_started":1}
W20260811 22:19:34.943305 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:34.933275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:58653 (local address 127.29.229.62:34795)
0811 22:19:34.933490 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:34.933495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:34.933968 (+ 473us) server_negotiation.cc:407] Connection header received
0811 22:19:34.934093 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:34.934098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:34.934203 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:34.934347 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:34.935232 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:34.936070 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:34.937551 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:34.937994 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:34.938174 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:34.942984 (+ 4810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.2:58653: BlockingRecv error: recv got EOF from 127.29.229.2:58653 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 22:19:35 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:35 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:35.286113 14426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:35.275706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.3:37069)
0811 22:19:35.276362 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:35.276379 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:35.276562 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:35.276934 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:35.276937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:35.277413 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:35.277643 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:35.277656 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:35.279002 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:35.279017 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:35.280113 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:35.280127 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:35.281071 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:35.281077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:35.281303 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:35.282039 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:35.282056 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:35.285860 (+ 3804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":388,"threads_started":1}
W20260811 22:19:35.286469 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:35.275804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:37069 (local address 127.29.229.62:34795)
0811 22:19:35.276071 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:35.276080 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:35.276516 (+ 436us) server_negotiation.cc:407] Connection header received
0811 22:19:35.276694 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:35.276698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:35.276772 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:35.276883 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:35.277852 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:35.278759 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:35.280230 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:35.280862 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:35.281011 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:35.286099 (+ 5088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.3:37069: BlockingRecv error: recv got EOF from 127.29.229.3:37069 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 11 22:19:35 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:35 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:35.498603 14427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:35.486449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.1:44193)
0811 22:19:35.486983 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:35.487001 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:35.487130 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:35.487562 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:35.487566 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:35.487998 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:35.488246 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:35.488263 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:35.489811 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:35.489818 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:35.491130 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:35.491145 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:35.492146 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:35.492152 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:35.492341 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:35.493231 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:35.493254 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:35.498294 (+ 5040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":302,"threads_started":1}
W20260811 22:19:35.498857 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:35.486547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:44193 (local address 127.29.229.62:34795)
0811 22:19:35.486838 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:35.486845 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:35.487178 (+ 333us) server_negotiation.cc:407] Connection header received
0811 22:19:35.487260 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:35.487265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:35.487346 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:35.487483 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:35.488405 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:35.489493 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:35.491356 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:35.491882 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:35.492042 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:35.498576 (+ 6534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.1:44193: BlockingRecv error: recv got EOF from 127.29.229.1:44193 (error 108)
Metrics: {"server-negotiator.queue_time_us":133}
Aug 11 22:19:35 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:35 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:35.955960 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:35.944234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.2:56351)
0811 22:19:35.945067 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:35.945089 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:35.945436 (+ 347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:35.946102 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:35.946106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:35.946600 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:35.946832 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:35.946842 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:35.947932 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:35.947939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:35.949524 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:35.949540 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:35.950330 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:35.950336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:35.950538 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:35.951275 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:35.951299 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:35.955679 (+ 4380us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":698,"thread_start_us":517,"threads_started":1}
W20260811 22:19:35.956297 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:35.944621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:56351 (local address 127.29.229.62:34795)
0811 22:19:35.944929 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:35.944937 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:35.945330 (+ 393us) server_negotiation.cc:407] Connection header received
0811 22:19:35.945597 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:35.945607 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:35.945677 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:35.945800 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:35.946985 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:35.947785 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:35.949684 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:35.950169 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:35.950274 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:35.955934 (+ 5660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.2:56351: BlockingRecv error: recv got EOF from 127.29.229.2:56351 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Aug 11 22:19:36 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:36 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:36.298091 14429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:36.287416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.3:57077)
0811 22:19:36.287996 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:36.288013 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:36.288132 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:36.288558 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:36.288562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:36.288966 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:36.289148 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:36.289158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:36.290349 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:36.290353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:36.291443 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:36.291458 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:36.292198 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:36.292203 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:36.292352 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:36.293181 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:36.293200 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:36.297800 (+ 4600us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":322,"threads_started":1}
W20260811 22:19:36.298463 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:36.287489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:57077 (local address 127.29.229.62:34795)
0811 22:19:36.287706 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:36.287711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:36.288126 (+ 415us) server_negotiation.cc:407] Connection header received
0811 22:19:36.288262 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:36.288266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:36.288330 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:36.288426 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:36.289334 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:36.290111 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:36.291602 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:36.292012 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:36.292115 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:36.298141 (+ 6026us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.3:57077: BlockingRecv error: recv got EOF from 127.29.229.3:57077 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 22:19:36 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:36 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:36.510214 14430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:36.499804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.1:58683)
0811 22:19:36.500464 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:36.500535 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:36.500718 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:36.501120 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:36.501125 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:36.501638 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:36.501797 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:36.501805 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:36.503129 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:36.503136 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:36.504456 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:36.504469 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:36.505198 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:36.505202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:36.505327 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:36.506075 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:36.506095 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:36.509848 (+ 3753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":340,"threads_started":1}
W20260811 22:19:36.510342 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:36.500006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:58683 (local address 127.29.229.62:34795)
0811 22:19:36.500314 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:36.500323 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:36.500638 (+ 315us) server_negotiation.cc:407] Connection header received
0811 22:19:36.500872 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:36.500878 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:36.500961 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:36.501081 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:36.501973 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:36.502969 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:36.504631 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:36.505045 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:36.505170 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:36.510060 (+ 4890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.1:58683: BlockingRecv error: recv got EOF from 127.29.229.1:58683 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
Aug 11 22:19:36 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:36 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:36.970525 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:36.957486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.2:54275)
0811 22:19:36.958472 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:36.958501 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:36.958645 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:36.959106 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:36.959110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:36.959644 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:36.959811 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:36.959830 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:36.961788 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:36.961797 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:36.963688 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:36.963708 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:36.964582 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:36.964590 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:36.964841 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:36.965585 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:36.965615 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:36.970180 (+ 4565us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":807,"thread_start_us":649,"threads_started":1}
W20260811 22:19:36.970664 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:36.957788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:54275 (local address 127.29.229.62:34795)
0811 22:19:36.958186 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:36.958195 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:36.958621 (+ 426us) server_negotiation.cc:407] Connection header received
0811 22:19:36.958785 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:36.958794 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:36.958897 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:36.959042 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:36.960009 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:36.961565 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:36.963910 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:36.964379 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:36.964513 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:36.970394 (+ 5881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.2:54275: BlockingRecv error: recv got EOF from 127.29.229.2:54275 (error 108)
Metrics: {"server-negotiator.queue_time_us":225}
Aug 11 22:19:37 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:37 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:37.310952 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:37.299651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.3:53907)
0811 22:19:37.300895 (+ 1244us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:37.300917 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:37.301063 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:37.301538 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:37.301545 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:37.302083 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:37.302298 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:37.302308 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:37.303597 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:37.303603 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:37.304745 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:37.304760 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:37.305761 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:37.305766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:37.305922 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:37.306662 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:37.306681 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:37.310687 (+ 4006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":970,"thread_start_us":549,"threads_started":1}
W20260811 22:19:37.311259 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:37.299734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:53907 (local address 127.29.229.62:34795)
0811 22:19:37.300073 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:37.300083 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:37.301019 (+ 936us) server_negotiation.cc:407] Connection header received
0811 22:19:37.301205 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:37.301211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:37.301303 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:37.301419 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:37.302502 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:37.303379 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:37.304897 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:37.305585 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:37.305715 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:37.310986 (+ 5271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.3:53907: BlockingRecv error: recv got EOF from 127.29.229.3:53907 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
Aug 11 22:19:37 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:37 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:37.521520 14437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:37.511407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.1:55147)
0811 22:19:37.511950 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:37.511966 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:37.512085 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:37.512426 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:37.512430 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:37.512942 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:37.513096 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:37.513105 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:37.514626 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:37.514634 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:37.515732 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:37.515746 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:37.516611 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:37.516616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:37.516744 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:37.517464 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:37.517482 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:37.521244 (+ 3762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":296,"threads_started":1}
W20260811 22:19:37.521739 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:37.511494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:55147 (local address 127.29.229.62:34795)
0811 22:19:37.511766 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:37.511771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:37.512077 (+ 306us) server_negotiation.cc:407] Connection header received
0811 22:19:37.512192 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:37.512195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:37.512278 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:37.512391 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:37.513252 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:37.514458 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:37.515894 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:37.516395 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:37.516581 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:37.521483 (+ 4902us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.1:55147: BlockingRecv error: recv got EOF from 127.29.229.1:55147 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 11 22:19:37 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:37 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:37.981752 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:37.971677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.2:56463)
0811 22:19:37.972248 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:37.972265 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:37.972435 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:37.972904 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:37.972909 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:37.973371 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:37.973542 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:37.973551 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:37.974591 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:37.974599 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:37.975633 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:37.975647 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:37.976644 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:37.976650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:37.976851 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:37.977682 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:37.977699 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:37.981508 (+ 3809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":332,"threads_started":1}
W20260811 22:19:37.981967 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:37.971780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:56463 (local address 127.29.229.62:34795)
0811 22:19:37.972012 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:37.972032 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:37.972365 (+ 333us) server_negotiation.cc:407] Connection header received
0811 22:19:37.972617 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:37.972623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:37.972736 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:37.972881 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:37.973693 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:37.974427 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:37.975791 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:37.976430 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:37.976627 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:37.981701 (+ 5074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.2:56463: BlockingRecv error: recv got EOF from 127.29.229.2:56463 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 11 22:19:38 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:38 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:38.321973 14441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:38.312162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.3:46681)
0811 22:19:38.312702 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:38.312726 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:38.312837 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:38.313199 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:38.313202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:38.313619 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:38.313789 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:38.313797 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:38.314877 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:38.314881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:38.315841 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:38.315853 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:38.316557 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:38.316562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:38.316692 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:38.317426 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:38.317452 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:38.321711 (+ 4259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":338,"threads_started":1}
W20260811 22:19:38.322166 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:38.312221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:46681 (local address 127.29.229.62:34795)
0811 22:19:38.312543 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:38.312552 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:38.312808 (+ 256us) server_negotiation.cc:407] Connection header received
0811 22:19:38.312993 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:38.312997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:38.313063 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:38.313156 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:38.313942 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:38.314704 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:38.315980 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:38.316389 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:38.316546 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:38.321943 (+ 5397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.3:46681: BlockingRecv error: recv got EOF from 127.29.229.3:46681 (error 108)
Metrics: {"server-negotiator.queue_time_us":161}
Aug 11 22:19:38 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:38 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:38.532290 14442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:38.522593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.1:55409)
0811 22:19:38.523173 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:38.523188 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:38.523350 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:38.523701 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:38.523706 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:38.524164 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:38.524368 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:38.524377 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:38.525434 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:38.525441 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:38.526400 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:38.526411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:38.527139 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:38.527145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:38.527316 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:38.527989 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:38.528015 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:38.531930 (+ 3915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":265,"threads_started":1}
W20260811 22:19:38.532413 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:38.522672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:55409 (local address 127.29.229.62:34795)
0811 22:19:38.522922 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:38.522930 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:38.523282 (+ 352us) server_negotiation.cc:407] Connection header received
0811 22:19:38.523478 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:38.523484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:38.523558 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:38.523666 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:38.524569 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:38.525293 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:38.526549 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:38.526984 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:38.527125 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:38.532180 (+ 5055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.1:55409: BlockingRecv error: recv got EOF from 127.29.229.1:55409 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 22:19:38 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:38 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:38.993067 14443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:38.983019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.2:53067)
0811 22:19:38.983698 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:38.983715 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:38.983837 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:38.984270 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:38.984274 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:38.984937 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:38.985174 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:38.985183 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:38.986578 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:38.986587 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:38.987578 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:38.987591 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:38.988355 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:38.988361 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:38.988597 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:38.989281 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:38.989299 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:38.992815 (+ 3516us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":308,"threads_started":1}
W20260811 22:19:38.993510 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:38.983118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:53067 (local address 127.29.229.62:34795)
0811 22:19:38.983370 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:38.983376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:38.983813 (+ 437us) server_negotiation.cc:407] Connection header received
0811 22:19:38.983955 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:38.983960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:38.984052 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:38.984178 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:38.985360 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:38.986395 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:38.987737 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:38.988181 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:38.988325 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:38.993064 (+ 4739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.2:53067: BlockingRecv error: recv got EOF from 127.29.229.2:53067 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
Aug 11 22:19:39 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:39 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:39.337775 14444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:39.323094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.3:59763)
0811 22:19:39.323739 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:39.323756 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:39.323875 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:39.324409 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:39.324415 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:39.325361 (+ 946us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:39.325559 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:39.325570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:39.326804 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:39.326808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:39.327812 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:39.327824 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:39.329012 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:39.329017 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:39.329164 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:39.333316 (+ 4152us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:39.333336 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:39.337534 (+ 4198us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":333,"threads_started":1}
W20260811 22:19:39.338157 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:39.323301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:59763 (local address 127.29.229.62:34795)
0811 22:19:39.323481 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:39.323486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:39.323861 (+ 375us) server_negotiation.cc:407] Connection header received
0811 22:19:39.323989 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:39.323993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:39.324063 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:39.324167 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:39.325744 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:39.326614 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:39.328041 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:39.328811 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:39.328937 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:39.337906 (+ 8969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.3:59763: BlockingRecv error: recv got EOF from 127.29.229.3:59763 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 22:19:39 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:39 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:39.545804 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:39.533978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.1:54301)
0811 22:19:39.534795 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:39.534828 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:39.535058 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:39.535447 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:39.535452 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:39.536061 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:39.536316 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:39.536332 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:39.537462 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:39.537471 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:39.538470 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:39.538481 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:39.539271 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:39.539277 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:39.539652 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:39.540566 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:39.540589 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:39.545386 (+ 4797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":413,"threads_started":1}
W20260811 22:19:39.545930 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:39.534402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:54301 (local address 127.29.229.62:34795)
0811 22:19:39.534744 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:39.534754 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:39.534964 (+ 210us) server_negotiation.cc:407] Connection header received
0811 22:19:39.535208 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:39.535214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:39.535291 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:39.535404 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:39.536520 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:39.537312 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:39.538642 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:39.539066 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:39.539200 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:39.545636 (+ 6436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.1:54301: BlockingRecv error: recv got EOF from 127.29.229.1:54301 (error 108)
Metrics: {"server-negotiator.queue_time_us":133}
Aug 11 22:19:40 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:40 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:40.007223 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:39.994496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.2:38401)
0811 22:19:39.995171 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:39.995188 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:39.995390 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:39.996121 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:39.996129 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:39.996974 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:39.997303 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:39.997318 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:39.999137 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:39.999142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:40.000413 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:40.000435 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.001462 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.001467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:40.001613 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:40.002331 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:40.002359 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:40.006942 (+ 4583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":325,"threads_started":1}
W20260811 22:19:40.007555 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:39.994719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:38401 (local address 127.29.229.62:34795)
0811 22:19:39.995024 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:39.995031 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:39.995412 (+ 381us) server_negotiation.cc:407] Connection header received
0811 22:19:39.995621 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:39.995635 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:39.995772 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:39.995959 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:39.997517 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:39.998890 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.000752 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.001204 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.001319 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:40.007264 (+ 5945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.2:38401: BlockingRecv error: recv got EOF from 127.29.229.2:38401 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
Aug 11 22:19:40 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:40 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:40.349807 14447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:40.338925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.3:34481)
0811 22:19:40.339527 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:40.339543 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:40.339718 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:40.340118 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:40.340125 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:40.340753 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:40.340927 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:40.340937 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.342062 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.342068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:40.343016 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:40.343028 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.343939 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.343945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:40.344128 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:40.344894 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:40.344912 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:40.349429 (+ 4517us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":332,"threads_started":1}
W20260811 22:19:40.350159 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:40.339081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:34481 (local address 127.29.229.62:34795)
0811 22:19:40.339335 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:40.339341 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:40.339643 (+ 302us) server_negotiation.cc:407] Connection header received
0811 22:19:40.339874 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:40.339880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:40.339960 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:40.340071 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:40.341120 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.341907 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.343171 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.343740 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.343886 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:40.349687 (+ 5801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.3:34481: BlockingRecv error: recv got EOF from 127.29.229.3:34481 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 11 22:19:40 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:40 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:40.557294 14449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:40.547020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.1:45131)
0811 22:19:40.547676 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:40.547696 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:40.547880 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:40.548509 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:40.548513 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:40.548971 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:40.549117 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:40.549125 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.550326 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.550330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:40.551291 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:40.551302 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.552036 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.552040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:40.552162 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:40.552915 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:40.552934 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:40.556899 (+ 3965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":388,"threads_started":1}
W20260811 22:19:40.557392 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:40.547035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:45131 (local address 127.29.229.62:34795)
0811 22:19:40.547241 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:40.547251 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:40.547956 (+ 705us) server_negotiation.cc:407] Connection header received
0811 22:19:40.548126 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:40.548132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:40.548256 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:40.548454 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:40.549288 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.550175 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.551454 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:40.551891 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:40.552005 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:40.557116 (+ 5111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.1:45131: BlockingRecv error: recv got EOF from 127.29.229.1:45131 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 22:19:41 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:41 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:41.019654 14450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:41.008584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.2:46237)
0811 22:19:41.009199 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:41.009217 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:41.009345 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:41.009893 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:41.009897 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:41.010327 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:41.010496 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:41.010504 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.011687 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.011692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:41.012762 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:41.012775 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.013599 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.013607 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:41.013754 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.014538 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:41.014563 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:41.019374 (+ 4811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":321,"threads_started":1}
W20260811 22:19:41.019858 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:41.008603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:46237 (local address 127.29.229.62:34795)
0811 22:19:41.008963 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:41.008970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:41.009364 (+ 394us) server_negotiation.cc:407] Connection header received
0811 22:19:41.009483 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:41.009488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:41.009671 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:41.009823 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:41.010695 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.011525 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.012920 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.013418 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.013531 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.019558 (+ 6027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.2:46237: BlockingRecv error: recv got EOF from 127.29.229.2:46237 (error 108)
Metrics: {"server-negotiator.queue_time_us":138}
Aug 11 22:19:41 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:41 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:41.362421 14453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:41.351289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.3:40637)
0811 22:19:41.351846 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:41.351864 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:41.351975 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:41.352962 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:41.352969 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:41.353606 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:41.353759 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:41.353769 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.355089 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.355093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:41.356108 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:41.356120 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.357128 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.357135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:41.357313 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.357996 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:41.358014 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:41.362064 (+ 4050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":291,"threads_started":1}
W20260811 22:19:41.362521 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:41.351358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:40637 (local address 127.29.229.62:34795)
0811 22:19:41.351569 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:41.351576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:41.351955 (+ 379us) server_negotiation.cc:407] Connection header received
0811 22:19:41.352371 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:41.352380 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:41.352555 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:41.352829 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:41.353980 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.354910 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.356315 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.356951 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.357093 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.362291 (+ 5198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.3:40637: BlockingRecv error: recv got EOF from 127.29.229.3:40637 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 22:19:41 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:41 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:41.569495 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:41.558404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.1:60051)
0811 22:19:41.558968 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:41.558985 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:41.559151 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:41.559561 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:41.559570 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:41.560128 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:41.560359 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:41.560371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.561664 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.561669 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:41.562688 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:41.562700 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.563760 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.563767 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:41.563966 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.564912 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:41.564937 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:41.569097 (+ 4160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":315,"threads_started":1}
W20260811 22:19:41.569602 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:41.558581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:60051 (local address 127.29.229.62:34795)
0811 22:19:41.558745 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:41.558750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:41.559120 (+ 370us) server_negotiation.cc:407] Connection header received
0811 22:19:41.559289 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:41.559295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:41.559393 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:41.559476 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:41.560680 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.561487 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.562879 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:41.563524 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:41.563674 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:41.569318 (+ 5644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.1:60051: BlockingRecv error: recv got EOF from 127.29.229.1:60051 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 11 22:19:42 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:42 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:42.030789 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:42.020800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.2:33471)
0811 22:19:42.021400 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:42.021418 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:42.021544 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:42.021910 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:42.021914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:42.022300 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:42.022474 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:42.022483 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:42.023607 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:42.023612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:42.024703 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:42.024715 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:42.025653 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:42.025660 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:42.025898 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:42.026658 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:42.026681 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:42.030526 (+ 3845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":345,"threads_started":1}
W20260811 22:19:42.030974 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:42.021004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:33471 (local address 127.29.229.62:34795)
0811 22:19:42.021188 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:42.021193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:42.021515 (+ 322us) server_negotiation.cc:407] Connection header received
0811 22:19:42.021651 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:42.021655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:42.021721 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:42.021814 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:42.022679 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:42.023414 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:42.024866 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:42.025434 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:42.025577 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:42.030711 (+ 5134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.2:33471: BlockingRecv error: recv got EOF from 127.29.229.2:33471 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 22:19:42 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:42 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:42.373489 14457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:42.363492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.3:37641)
0811 22:19:42.364388 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:42.364405 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:42.364601 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:42.364976 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:42.364981 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:42.365395 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:42.365577 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:42.365587 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:42.366660 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:42.366665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:42.367703 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:42.367716 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:42.368653 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:42.368660 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:42.368832 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:42.369502 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:42.369526 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:42.373165 (+ 3639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":791,"thread_start_us":624,"threads_started":1}
W20260811 22:19:42.373699 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:42.363745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:37641 (local address 127.29.229.62:34795)
0811 22:19:42.364259 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:42.364266 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:42.364505 (+ 239us) server_negotiation.cc:407] Connection header received
0811 22:19:42.364740 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:42.364745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:42.364833 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:42.364945 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:42.365730 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:42.366503 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:42.367859 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:42.368439 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:42.368627 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:42.373440 (+ 4813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.3:37641: BlockingRecv error: recv got EOF from 127.29.229.3:37641 (error 108)
Metrics: {"server-negotiator.queue_time_us":211}
Aug 11 22:19:42 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:42 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:42.582218 14458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:42.570907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.1:58293)
0811 22:19:42.571780 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:42.571801 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:42.571929 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:42.572272 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:42.572275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:42.572815 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:42.573066 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:42.573076 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:42.574419 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:42.574423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:42.575697 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:42.575715 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:42.577000 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:42.577010 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:42.577359 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:42.578178 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:42.578204 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:42.581976 (+ 3772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":527,"threads_started":1}
W20260811 22:19:42.582468 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:42.571665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:58293 (local address 127.29.229.62:34795)
0811 22:19:42.571906 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:42.571911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:42.571930 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:19:42.572024 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:42.572028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:42.572128 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:42.572218 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:42.573229 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:42.574248 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:42.575872 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:42.576753 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:42.576920 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:42.582206 (+ 5286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.1:58293: BlockingRecv error: recv got EOF from 127.29.229.1:58293 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 11 22:19:43 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:43 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:43.041572 14460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:43.031883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.2:53149)
0811 22:19:43.032383 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:43.032401 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:43.032551 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:43.032864 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:43.032867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:43.033306 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:43.033492 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:43.033500 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:43.034561 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:43.034566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:43.035500 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:43.035512 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:43.036243 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:43.036248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:43.036387 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:43.037108 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:43.037127 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:43.041209 (+ 4082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":278,"threads_started":1}
W20260811 22:19:43.041759 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:43.031963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:53149 (local address 127.29.229.62:34795)
0811 22:19:43.032216 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:43.032223 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:43.032516 (+ 293us) server_negotiation.cc:407] Connection header received
0811 22:19:43.032662 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:43.032666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:43.032731 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:43.032835 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:43.033629 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:43.034420 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:43.035647 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:43.036067 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:43.036165 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:43.041488 (+ 5323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.2:53149: BlockingRecv error: recv got EOF from 127.29.229.2:53149 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 22:19:43 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:43 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:43.386445 14461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:43.374673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.3:57271)
0811 22:19:43.375256 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:43.375282 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:43.375451 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:43.375912 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:43.375915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:43.376498 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:43.376694 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:43.376703 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:43.377990 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:43.377995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:43.379026 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:43.379038 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:43.380412 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:43.380419 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:43.380657 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:43.381410 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:43.381434 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:43.386069 (+ 4635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":365,"threads_started":1}
W20260811 22:19:43.386643 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:43.374783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:57271 (local address 127.29.229.62:34795)
0811 22:19:43.374974 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:43.374978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:43.375375 (+ 397us) server_negotiation.cc:407] Connection header received
0811 22:19:43.375579 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:43.375583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:43.375704 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:43.375844 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:43.376876 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:43.377829 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:43.379246 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:43.380015 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:43.380168 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:43.386324 (+ 6156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.3:57271: BlockingRecv error: recv got EOF from 127.29.229.3:57271 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 22:19:43 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:43 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:43.592810 14462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:43.583296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.1:59467)
0811 22:19:43.583857 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:43.583881 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:43.584049 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:43.584407 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:43.584412 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:43.584895 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:43.585074 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:43.585083 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:43.586120 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:43.586127 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:43.587058 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:43.587069 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:43.587856 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:43.587862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:43.588059 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:43.588747 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:43.588765 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:43.592398 (+ 3633us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":284,"threads_started":1}
W20260811 22:19:43.592931 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:43.583389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:59467 (local address 127.29.229.62:34795)
0811 22:19:43.583637 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:43.583645 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:43.583952 (+ 307us) server_negotiation.cc:407] Connection header received
0811 22:19:43.584189 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:43.584193 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:43.584268 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:43.584375 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:43.585228 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:43.585976 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:43.587205 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:43.587695 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:43.587838 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:43.592660 (+ 4822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.1:59467: BlockingRecv error: recv got EOF from 127.29.229.1:59467 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 22:19:44 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:44 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:44.053574 14463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:44.042834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.2:40541)
0811 22:19:44.043426 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:44.043443 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:44.043570 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:44.043915 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:44.043919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:44.044360 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:44.044659 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:44.044670 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:44.045742 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:44.045746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:44.046905 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:44.046918 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:44.047978 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:44.047987 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:44.048214 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:44.048929 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:44.048949 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:44.053284 (+ 4335us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":357,"threads_started":1}
W20260811 22:19:44.053812 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:44.042914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:40541 (local address 127.29.229.62:34795)
0811 22:19:44.043173 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:44.043183 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:44.043559 (+ 376us) server_negotiation.cc:407] Connection header received
0811 22:19:44.043688 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:44.043692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:44.043759 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:44.043852 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:44.044826 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:44.045588 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:44.047106 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:44.047781 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:44.047929 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:44.053543 (+ 5614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.2:40541: BlockingRecv error: recv got EOF from 127.29.229.2:40541 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 22:19:44 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:44 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:44.402834 14464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:44.389010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.3:42937)
0811 22:19:44.389627 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:44.389644 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:44.389766 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:44.390380 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:44.390384 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:44.390779 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:44.390995 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:44.391005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:44.392254 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:44.392260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:44.393515 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:44.393532 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:44.394452 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:44.394458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:44.394636 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:44.398368 (+ 3732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:44.398390 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:44.402561 (+ 4171us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":373,"threads_started":1}
W20260811 22:19:44.403072 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:44.389068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:42937 (local address 127.29.229.62:34795)
0811 22:19:44.389304 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:44.389315 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:44.389730 (+ 415us) server_negotiation.cc:407] Connection header received
0811 22:19:44.390032 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:44.390038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:44.390160 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:44.390304 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:44.391206 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:44.392000 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:44.393713 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:44.394197 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:44.394327 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:44.402805 (+ 8478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.3:42937: BlockingRecv error: recv got EOF from 127.29.229.3:42937 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 22:19:44 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:44 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:44.604002 14465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:44.594147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.1:39981)
0811 22:19:44.594646 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:44.594663 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:44.594789 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:44.595156 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:44.595160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:44.595602 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:44.595753 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:44.595761 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:44.596996 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:44.597002 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:44.598064 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:44.598080 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:44.599124 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:44.599130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:44.599328 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:44.600001 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:44.600019 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:44.603724 (+ 3705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":281,"threads_started":1}
W20260811 22:19:44.604292 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:44.594301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:39981 (local address 127.29.229.62:34795)
0811 22:19:44.594576 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:44.594582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:44.594745 (+ 163us) server_negotiation.cc:407] Connection header received
0811 22:19:44.594884 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:44.594888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:44.594971 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:44.595064 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:44.595906 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:44.596782 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:44.598283 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:44.598955 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:44.599097 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:44.603994 (+ 4897us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.1:39981: BlockingRecv error: recv got EOF from 127.29.229.1:39981 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 11 22:19:45 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:45 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.2@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:45.066287 14466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:45.055274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.2:54963)
0811 22:19:45.056292 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:45.056317 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:45.056582 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:45.057027 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:45.057034 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:45.057735 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:45.057948 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:45.057962 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:45.059143 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:45.059148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:45.060112 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:45.060124 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:45.061003 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:45.061011 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:45.061198 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:45.061905 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:45.061923 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:45.066033 (+ 4110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":760,"thread_start_us":560,"threads_started":1}
W20260811 22:19:45.066529 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:45.055566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:54963 (local address 127.29.229.62:34795)
0811 22:19:45.055853 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:45.055862 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:45.056449 (+ 587us) server_negotiation.cc:407] Connection header received
0811 22:19:45.056747 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:45.056754 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:45.056843 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:45.056997 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:45.058181 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:45.058980 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:45.060270 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:45.060831 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:45.060966 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:45.066270 (+ 5304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.2:54963: BlockingRecv error: recv got EOF from 127.29.229.2:54963 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
Aug 11 22:19:45 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:45 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.3@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:45.419950 14467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:45.404472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.3:42483)
0811 22:19:45.406492 (+ 2020us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:45.406510 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:45.406700 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:45.407297 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:45.407302 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:45.407941 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:45.408176 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:45.408189 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:45.409364 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:45.409373 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:45.410712 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:45.410727 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:45.411583 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:45.411589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:45.411772 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:45.412829 (+ 1057us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:45.412864 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:45.419566 (+ 6702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1892,"thread_start_us":1771,"threads_started":1}
W20260811 22:19:45.420269 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:45.406016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:42483 (local address 127.29.229.62:34795)
0811 22:19:45.406265 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:45.406274 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:45.406623 (+ 349us) server_negotiation.cc:407] Connection header received
0811 22:19:45.406844 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:45.406850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:45.406926 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:45.407045 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:45.408338 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:45.409211 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:45.410889 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:45.411418 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:45.411567 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:45.419838 (+ 8271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.3:42483: BlockingRecv error: recv got EOF from 127.29.229.3:42483 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 11 22:19:45 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:19:45 dist-test-slave-jq4t krb5kdc[12749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.229.1@KRBTEST.COM for oryx/127.29.229.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:19:45.616307 14468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:45.605430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:34795 (local address 127.29.229.1:34487)
0811 22:19:45.605964 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:45.605980 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:45.606179 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:45.606568 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:45.606571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:45.606974 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:45.607154 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:45.607163 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:45.608285 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:45.608290 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:45.609463 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:45.609479 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:45.610350 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:45.610355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:45.610493 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:45.611211 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:45.611231 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:45.615810 (+ 4579us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.229.62:34795: Server oryx/127.29.229.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":248,"threads_started":1}
W20260811 22:19:45.616431 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:45.605594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:34487 (local address 127.29.229.62:34795)
0811 22:19:45.605943 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:45.605950 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:45.606064 (+ 114us) server_negotiation.cc:407] Connection header received
0811 22:19:45.606328 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:45.606333 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:45.606419 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:45.606522 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:45.607326 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:45.608121 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:45.609689 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:45.610188 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:45.610299 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:45.616106 (+ 5807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.229.1:34487: BlockingRecv error: recv got EOF from 127.29.229.1:34487 (error 108)
Metrics: {"server-negotiator.queue_time_us":203}
I20260811 22:19:45.959539 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 13747
I20260811 22:19:45.973592 14471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:45.961007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34966 (local address 127.29.229.1:46561)
0811 22:19:45.961726 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:45.961734 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:45.961765 (+ 31us) server_negotiation.cc:407] Connection header received
0811 22:19:45.961848 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:45.961854 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:45.962130 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:45.962294 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:45.963808 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:45.965035 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:45.966324 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:45.967127 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:45.967243 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:45.969905 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:45.969913 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:45.969927 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:45.969976 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:45.972808 (+ 2832us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.229.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 22:19:45.973231 (+ 423us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34966: Request ticket server kudu/127.29.229.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":384,"threads_started":1}
W20260811 22:19:45.973634 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 13747: Not authorized: Client connection negotiation failed: client connection to 127.29.229.1:46561: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.229.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 22:19:45.973865 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 13747
W20260811 22:19:45.973843 14471 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34966: Request ticket server kudu/127.29.229.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 22:19:46.008920 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 14000
I20260811 22:19:46.022717 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:46.010255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32916 (local address 127.29.229.2:33637)
0811 22:19:46.010877 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:46.010884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:46.010910 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:19:46.010976 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:46.010983 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:46.011240 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:46.011463 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:46.012975 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:46.014024 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:46.015276 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:46.015870 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:46.016021 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:46.019021 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:46.019030 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:46.019045 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:46.019085 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:46.021921 (+ 2836us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.229.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 22:19:46.022331 (+ 410us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:32916: Request ticket server kudu/127.29.229.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":319,"threads_started":1}
W20260811 22:19:46.022717 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 14000: Not authorized: Client connection negotiation failed: client connection to 127.29.229.2:33637: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.229.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 22:19:46.022876 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 14000
W20260811 22:19:46.022838 14476 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:32916: Request ticket server kudu/127.29.229.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 22:19:46.050873 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 14202
I20260811 22:19:46.068403 14477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:46.052468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35736 (local address 127.29.229.3:46151)
0811 22:19:46.053601 (+ 1133us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:46.053611 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:46.053647 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:19:46.053742 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:46.053749 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:46.054077 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:46.054292 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:46.057093 (+ 2801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:46.058761 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:46.060154 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:46.061019 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:46.061330 (+ 311us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:46.064324 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:46.064334 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:46.064352 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:46.064401 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:46.067759 (+ 3358us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.229.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 22:19:46.068105 (+ 346us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35736: Request ticket server kudu/127.29.229.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":850,"thread_start_us":656,"threads_started":1}
W20260811 22:19:46.068554 14477 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35736: Request ticket server kudu/127.29.229.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 22:19:46.068632 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 14202: Not authorized: Client connection negotiation failed: client connection to 127.29.229.3:46151: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.229.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 22:19:46.068830 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 14202
I20260811 22:19:46.100540 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 13639
I20260811 22:19:46.102080 13733 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:46.230367 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 13639
2026-08-11T22:19:46Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22334 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T22:19:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:19:46Z Disabled control of system clock
I20260811 22:19:46.350965 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:33777
--webserver_interface=127.29.229.62
--webserver_port=0
--builtin_ntp_servers=127.29.229.20:45651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:33777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 22:19:46.735730 14487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:46.736052 14487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:46.736135 14487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:46.743773 14487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:19:46.743914 14487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:46.743959 14487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:46.744014 14487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:19:46.744053 14487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:19:46.753954 14487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:45651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:33777
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:33777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:46.755347 14487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:46.757232 14487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:46.770961 14494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:46.770854 14498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:46.770895 14502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:46.773231 14487 server_base.cc:1034] running on GCE node
I20260811 22:19:46.774458 14487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:46.776264 14487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:46.777587 14487 hybrid_clock.cc:648] HybridClock initialized: now 1786486786777500 us; error 114 us; skew 500 ppm
I20260811 22:19:46.778383 14487 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:19:46.781694 14487 webserver.cc:460] Webserver started at http://127.29.229.62:44293/ using document root <none> and password file <none>
I20260811 22:19:46.782546 14487 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:46.782703 14487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:46.783084 14487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:46.786381 14487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "59c930faf2ee46d0b68fc5a0e9485381"
format_stamp: "Formatted at 2026-08-11 22:19:46 on dist-test-slave-jq4t"
server_key: "a790411af5685af1cd611dd3f5618aff"
server_key_iv: "79266dbd830d1bec087d01c3baa30775"
server_key_version: "encryptionkey@0"
I20260811 22:19:46.787339 14487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "59c930faf2ee46d0b68fc5a0e9485381"
format_stamp: "Formatted at 2026-08-11 22:19:46 on dist-test-slave-jq4t"
server_key: "a790411af5685af1cd611dd3f5618aff"
server_key_iv: "79266dbd830d1bec087d01c3baa30775"
server_key_version: "encryptionkey@0"
I20260811 22:19:46.793792 14487 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 22:19:46.798504 14513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:46.799993 14487 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:19:46.800217 14487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "59c930faf2ee46d0b68fc5a0e9485381"
format_stamp: "Formatted at 2026-08-11 22:19:46 on dist-test-slave-jq4t"
server_key: "a790411af5685af1cd611dd3f5618aff"
server_key_iv: "79266dbd830d1bec087d01c3baa30775"
server_key_version: "encryptionkey@0"
I20260811 22:19:46.800518 14487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:46.813791 14487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:46.814842 14487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:46.815163 14487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:46.836256 14487 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:33777
I20260811 22:19:46.836280 14612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:33777 every 8 connection(s)
I20260811 22:19:46.837741 14487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:19:46.838337 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 14487
I20260811 22:19:46.838585 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:19:46.839478 30612 external_mini_cluster.cc:1428] Setting key 8dba6b30df4270dbe74b37f9df4ba0d5
I20260811 22:19:46.842339 14613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:46.854278 14617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:46.841687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44598 (local address 127.29.229.62:33777)
0811 22:19:46.842740 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:46.842751 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:46.842796 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:19:46.844617 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:46.844641 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:46.845124 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:46.845555 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:46.846198 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:46.847668 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:46.849240 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:46.849977 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:46.850189 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:46.851644 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:46.851653 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:46.851669 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:19:46.852340 (+ 671us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:46.852430 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:19:46.852455 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:46.852669 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:46.852931 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 22:19:46.853269 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":346,"threads_started":1}
I20260811 22:19:46.855104 14613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381: Bootstrap starting.
I20260811 22:19:46.859212 14613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:46.860321 14613 log.cc:824] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:46.863901 14613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381: No bootstrap required, opened a new log
I20260811 22:19:46.872344 14613 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59c930faf2ee46d0b68fc5a0e9485381" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 33777 } }
I20260811 22:19:46.872812 14613 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:46.872921 14613 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59c930faf2ee46d0b68fc5a0e9485381, State: Initialized, Role: FOLLOWER
I20260811 22:19:46.873533 14613 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59c930faf2ee46d0b68fc5a0e9485381" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 33777 } }
I20260811 22:19:46.873745 14613 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:19:46.873850 14613 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:19:46.873987 14613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:46.876681 14613 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59c930faf2ee46d0b68fc5a0e9485381" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 33777 } }
I20260811 22:19:46.877154 14613 leader_election.cc:302] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59c930faf2ee46d0b68fc5a0e9485381; no voters:
I20260811 22:19:46.877704 14613 leader_election.cc:288] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:19:46.877873 14619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:19:46.878636 14619 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381 [term 1 LEADER]: Becoming Leader. State: Replica: 59c930faf2ee46d0b68fc5a0e9485381, State: Running, Role: LEADER
I20260811 22:19:46.879148 14619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59c930faf2ee46d0b68fc5a0e9485381" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 33777 } }
I20260811 22:19:46.880049 14613 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:19:46.882028 14620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59c930faf2ee46d0b68fc5a0e9485381" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59c930faf2ee46d0b68fc5a0e9485381" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 33777 } } }
I20260811 22:19:46.882745 14620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:46.882568 14621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59c930faf2ee46d0b68fc5a0e9485381. Latest consensus state: current_term: 1 leader_uuid: "59c930faf2ee46d0b68fc5a0e9485381" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59c930faf2ee46d0b68fc5a0e9485381" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 33777 } } }
I20260811 22:19:46.882864 14621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:46.887950 14627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:19:46.895249 14627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:19:46.906374 14627 catalog_manager.cc:1347] Generated new cluster ID: 2e42131684c64e50934f6579f0809172
I20260811 22:19:46.906514 14627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:19:46.926645 14627 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:19:46.927994 14627 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:19:46.942950 14627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59c930faf2ee46d0b68fc5a0e9485381: Generated new TSK 0
I20260811 22:19:46.943910 14627 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:19:46.957852 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:33777
--builtin_ntp_servers=127.29.229.20:45651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 22:19:47.350036 14654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:47.350347 14654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:47.350445 14654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:47.358381 14654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:47.358510 14654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:47.358675 14654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:19:47.368469 14654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:45651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:33777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:47.369930 14654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:47.372964 14654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:47.393142 14669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:47.393144 14668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:47.393172 14674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:47.394299 14654 server_base.cc:1034] running on GCE node
I20260811 22:19:47.395094 14654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:47.396312 14654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:47.397564 14654 hybrid_clock.cc:648] HybridClock initialized: now 1786486787397494 us; error 59 us; skew 500 ppm
I20260811 22:19:47.397953 14654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:19:47.400377 14654 webserver.cc:460] Webserver started at http://127.29.229.1:36859/ using document root <none> and password file <none>
I20260811 22:19:47.401175 14654 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:47.401291 14654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:47.401645 14654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:47.404677 14654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "c311ec9c1b524b40a1887cc8b495a9ad"
format_stamp: "Formatted at 2026-08-11 22:19:47 on dist-test-slave-jq4t"
server_key: "e59e415d596bb56f7f4f90d951dec559"
server_key_iv: "4f2b7aab19db83ab9be66d2c47cba232"
server_key_version: "encryptionkey@0"
I20260811 22:19:47.405494 14654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "c311ec9c1b524b40a1887cc8b495a9ad"
format_stamp: "Formatted at 2026-08-11 22:19:47 on dist-test-slave-jq4t"
server_key: "e59e415d596bb56f7f4f90d951dec559"
server_key_iv: "4f2b7aab19db83ab9be66d2c47cba232"
server_key_version: "encryptionkey@0"
I20260811 22:19:47.411383 14654 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:19:47.415162 14689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:47.416719 14654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:19:47.416915 14654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "c311ec9c1b524b40a1887cc8b495a9ad"
format_stamp: "Formatted at 2026-08-11 22:19:47 on dist-test-slave-jq4t"
server_key: "e59e415d596bb56f7f4f90d951dec559"
server_key_iv: "4f2b7aab19db83ab9be66d2c47cba232"
server_key_version: "encryptionkey@0"
I20260811 22:19:47.417140 14654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:47.432888 14654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:47.433772 14654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:47.434296 14654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:47.435567 14654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:47.437286 14654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:47.437431 14654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:47.437561 14654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:47.437661 14654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:47.474772 14654 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:42545
I20260811 22:19:47.474838 14881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:42545 every 8 connection(s)
I20260811 22:19:47.476244 14654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:19:47.479169 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 14654
I20260811 22:19:47.479393 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:19:47.480158 30612 external_mini_cluster.cc:1428] Setting key cfb46b7773419f455565baf37bf4ef73
I20260811 22:19:47.483264 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:33777
--builtin_ntp_servers=127.29.229.20:45651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 22:19:47.492700 14887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:47.479248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:54905 (local address 127.29.229.62:33777)
0811 22:19:47.479841 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:47.479847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:47.481311 (+ 1464us) server_negotiation.cc:407] Connection header received
0811 22:19:47.483700 (+ 2389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:47.483709 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:47.483814 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:47.483899 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:47.485775 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:47.486513 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:47.489233 (+ 2720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:47.489721 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:47.489844 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:47.491711 (+ 1867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:47.491716 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:47.491719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:19:47.491740 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:47.491781 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:19:47.491793 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:47.491880 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:47.492233 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 22:19:47.492337 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":330,"threads_started":1}
I20260811 22:19:47.494176 14888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:47.479939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:33777 (local address 127.29.229.1:54905)
0811 22:19:47.481147 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:47.481193 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:47.483341 (+ 2148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:47.484448 (+ 1107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:47.484462 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:47.484634 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:47.485549 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:47.485580 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:47.486710 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:47.486718 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:47.488982 (+ 2264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:47.489002 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:47.489972 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:47.489982 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:47.490335 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:47.491264 (+ 929us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:19:47.491269 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:47.491330 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:19:47.491445 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:47.491906 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:47.491919 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:47.491931 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 22:19:47.492129 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 22:19:47.492660 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":421,"threads_started":1}
I20260811 22:19:47.495936 14885 heartbeater.cc:347] Connected to a master server at 127.29.229.62:33777
I20260811 22:19:47.496273 14885 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:47.497319 14885 heartbeater.cc:511] Master 127.29.229.62:33777 requested a full tablet report, sending...
I20260811 22:19:47.499554 14543 ts_manager.cc:194] Registered new tserver with Master: c311ec9c1b524b40a1887cc8b495a9ad (127.29.229.1:42545)
I20260811 22:19:47.501121 14543 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.229.1:54905
W20260811 22:19:47.850816 14889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:47.851110 14889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:47.851267 14889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:47.859442 14889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:47.859573 14889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:47.859761 14889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:19:47.869959 14889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:45651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:33777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:47.871348 14889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:47.873256 14889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:47.886656 14896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:47.886695 14901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:47.886718 14904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:47.888178 14889 server_base.cc:1034] running on GCE node
I20260811 22:19:47.888902 14889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:47.889976 14889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:47.891183 14889 hybrid_clock.cc:648] HybridClock initialized: now 1786486787891130 us; error 43 us; skew 500 ppm
I20260811 22:19:47.891512 14889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:19:47.894011 14889 webserver.cc:460] Webserver started at http://127.29.229.2:38003/ using document root <none> and password file <none>
I20260811 22:19:47.894767 14889 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:47.894865 14889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:47.895239 14889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:47.898712 14889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "81b383bd67e34a89b9ee52bc4adc9277"
format_stamp: "Formatted at 2026-08-11 22:19:47 on dist-test-slave-jq4t"
server_key: "9713974b157736feb0b8978425b65561"
server_key_iv: "4a6a273544163bf78ea44186bb7a8640"
server_key_version: "encryptionkey@0"
I20260811 22:19:47.899582 14889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "81b383bd67e34a89b9ee52bc4adc9277"
format_stamp: "Formatted at 2026-08-11 22:19:47 on dist-test-slave-jq4t"
server_key: "9713974b157736feb0b8978425b65561"
server_key_iv: "4a6a273544163bf78ea44186bb7a8640"
server_key_version: "encryptionkey@0"
I20260811 22:19:47.905186 14889 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 22:19:47.909046 14913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:47.910439 14889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:19:47.910622 14889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "81b383bd67e34a89b9ee52bc4adc9277"
format_stamp: "Formatted at 2026-08-11 22:19:47 on dist-test-slave-jq4t"
server_key: "9713974b157736feb0b8978425b65561"
server_key_iv: "4a6a273544163bf78ea44186bb7a8640"
server_key_version: "encryptionkey@0"
I20260811 22:19:47.910885 14889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:47.946527 14889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:47.947476 14889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:47.947813 14889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:47.948956 14889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:47.950467 14889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:47.950608 14889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:47.950732 14889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:47.950837 14889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:47.987323 14889 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:39925
I20260811 22:19:47.987423 15098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:39925 every 8 connection(s)
I20260811 22:19:47.988901 14889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:19:47.989645 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 14889
I20260811 22:19:47.989845 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:19:47.990479 30612 external_mini_cluster.cc:1428] Setting key bd39bd613f5d1cd49a92bdae0f9c7f4b
I20260811 22:19:47.994146 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:33777
--builtin_ntp_servers=127.29.229.20:45651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 22:19:48.004001 14887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:47.992754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:50063 (local address 127.29.229.62:33777)
0811 22:19:47.992994 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:47.993000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:47.994881 (+ 1881us) server_negotiation.cc:407] Connection header received
0811 22:19:47.996803 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:47.996808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:47.996900 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:47.997015 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:47.998065 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:47.998790 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:48.000339 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:48.000844 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:48.000976 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:48.002782 (+ 1806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:48.002789 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:48.002792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:19:48.002819 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:48.002863 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:19:48.002877 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:48.002991 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:48.003272 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 22:19:48.003420 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 22:19:48.005339 15102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:47.993570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:33777 (local address 127.29.229.2:50063)
0811 22:19:47.994738 (+ 1168us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:47.994776 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:47.996550 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:47.997241 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:47.997250 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:47.997339 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:47.997903 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:47.997919 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:47.998946 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:47.998952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:48.000196 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:48.000209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:48.001079 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:48.001087 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:48.001314 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:48.002193 (+ 879us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:19:48.002198 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:48.002259 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:19:48.002394 (+ 135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:48.002996 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:48.003013 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:48.003023 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 22:19:48.003260 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 22:19:48.003955 (+ 695us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":384,"threads_started":1}
I20260811 22:19:48.006991 15099 heartbeater.cc:347] Connected to a master server at 127.29.229.62:33777
I20260811 22:19:48.007362 15099 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:48.008183 15099 heartbeater.cc:511] Master 127.29.229.62:33777 requested a full tablet report, sending...
I20260811 22:19:48.010327 14543 ts_manager.cc:194] Registered new tserver with Master: 81b383bd67e34a89b9ee52bc4adc9277 (127.29.229.2:39925)
I20260811 22:19:48.011170 14543 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.229.2:50063
W20260811 22:19:48.364605 15105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:48.364936 15105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:48.365089 15105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:48.374547 15105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:48.374791 15105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:48.374994 15105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:19:48.386754 15105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:45651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:33777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:48.388207 15105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:48.390283 15105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:48.405364 15116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:48.405416 15117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:48.405611 15120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:48.407176 15105 server_base.cc:1034] running on GCE node
I20260811 22:19:48.407907 15105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:48.409212 15105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:48.410457 15105 hybrid_clock.cc:648] HybridClock initialized: now 1786486788410391 us; error 62 us; skew 500 ppm
I20260811 22:19:48.410811 15105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:19:48.413297 15105 webserver.cc:460] Webserver started at http://127.29.229.3:38455/ using document root <none> and password file <none>
I20260811 22:19:48.414076 15105 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:48.414214 15105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:48.414597 15105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:48.417718 15105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "0a1dbc4fd2574f6e8ce0f65299e571d8"
format_stamp: "Formatted at 2026-08-11 22:19:48 on dist-test-slave-jq4t"
server_key: "76d597fff135b7aecba7a579185c9b9e"
server_key_iv: "a7465c8a5a16f32ccae824ca02ca177b"
server_key_version: "encryptionkey@0"
I20260811 22:19:48.418578 15105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "0a1dbc4fd2574f6e8ce0f65299e571d8"
format_stamp: "Formatted at 2026-08-11 22:19:48 on dist-test-slave-jq4t"
server_key: "76d597fff135b7aecba7a579185c9b9e"
server_key_iv: "a7465c8a5a16f32ccae824ca02ca177b"
server_key_version: "encryptionkey@0"
I20260811 22:19:48.424028 15105 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 22:19:48.427678 15131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:48.429177 15105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:19:48.429347 15105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "0a1dbc4fd2574f6e8ce0f65299e571d8"
format_stamp: "Formatted at 2026-08-11 22:19:48 on dist-test-slave-jq4t"
server_key: "76d597fff135b7aecba7a579185c9b9e"
server_key_iv: "a7465c8a5a16f32ccae824ca02ca177b"
server_key_version: "encryptionkey@0"
I20260811 22:19:48.429577 15105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:48.450978 15105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:48.451861 15105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:48.452188 15105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:48.453866 15105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:48.456073 15105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:48.456380 15105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:48.456566 15105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:48.456664 15105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:48.496729 15105 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:41715
I20260811 22:19:48.496765 15325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:41715 every 8 connection(s)
I20260811 22:19:48.498116 15105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:19:48.500677 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 15105
I20260811 22:19:48.500860 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:19:48.501538 30612 external_mini_cluster.cc:1428] Setting key 5cffbdd5db1f9d84e18d8f533276b1b4
I20260811 22:19:48.505923 14885 heartbeater.cc:503] Master 127.29.229.62:33777 was elected leader, sending a full tablet report...
I20260811 22:19:48.515571 14887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:48.500922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:50643 (local address 127.29.229.62:33777)
0811 22:19:48.501145 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:48.501151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:48.502831 (+ 1680us) server_negotiation.cc:407] Connection header received
0811 22:19:48.505217 (+ 2386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:48.505223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:48.505322 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:48.505437 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:48.506945 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:48.508781 (+ 1836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:48.511853 (+ 3072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:48.512405 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:48.512604 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:48.514573 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:48.514578 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:48.514581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:19:48.514602 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:48.514636 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:19:48.514655 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:48.514745 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:48.515056 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 22:19:48.515204 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 22:19:48.517231 15329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:48.501568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:33777 (local address 127.29.229.3:50643)
0811 22:19:48.502683 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:48.502726 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:48.504869 (+ 2143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:48.505689 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:48.505701 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:48.505833 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:48.506732 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:48.506762 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:48.509062 (+ 2300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:48.509071 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:48.511617 (+ 2546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:48.511639 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:48.512731 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:48.512742 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:48.513136 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:48.514046 (+ 910us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:19:48.514052 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:48.514123 (+ 71us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:19:48.514265 (+ 142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:48.514795 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:48.514809 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:48.514821 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 22:19:48.515056 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 22:19:48.515570 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":373,"threads_started":1}
I20260811 22:19:48.518918 15327 heartbeater.cc:347] Connected to a master server at 127.29.229.62:33777
I20260811 22:19:48.519259 15327 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:48.520094 15327 heartbeater.cc:511] Master 127.29.229.62:33777 requested a full tablet report, sending...
I20260811 22:19:48.521812 14543 ts_manager.cc:194] Registered new tserver with Master: 0a1dbc4fd2574f6e8ce0f65299e571d8 (127.29.229.3:41715)
I20260811 22:19:48.522753 14543 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.229.3:50643
I20260811 22:19:48.531059 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:19:48.538488 14887 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:48.536633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44606 (local address 127.29.229.62:33777)
0811 22:19:48.536875 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:48.536884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:48.537439 (+ 555us) server_negotiation.cc:407] Connection header received
0811 22:19:48.537632 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:48.537635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:48.537700 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:48.537782 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:48.538230 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44606: BlockingRecv error: recv got EOF from 127.0.0.1:44606 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:19:48.539541 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 14654
I20260811 22:19:48.548079 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:48.540710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37994 (local address 127.29.229.1:42545)
0811 22:19:48.541328 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:48.541335 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:48.541361 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:19:48.541429 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:48.541436 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:48.541653 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:48.541824 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:48.542271 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:48.543332 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:48.544667 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:48.545272 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:48.545436 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:48.546600 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:48.546608 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:48.546622 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:19:48.547215 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:48.547262 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:19:48.547290 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:48.547416 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:48.547575 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 22:19:48.547720 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":348,"threads_started":1}
I20260811 22:19:48.549543 14873 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:48.636600 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 14654
I20260811 22:19:48.664919 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 14889
I20260811 22:19:48.673542 15347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:48.665942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53100 (local address 127.29.229.2:39925)
0811 22:19:48.666569 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:48.666576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:48.666605 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:19:48.666670 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:48.666676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:48.666905 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:48.667046 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:48.667487 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:48.668569 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:48.669841 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:48.670484 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:48.670593 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:48.671844 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:48.671853 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:48.671867 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:19:48.672630 (+ 763us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:48.672682 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:19:48.672705 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:48.672802 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:48.673003 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 22:19:48.673128 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":339,"threads_started":1}
I20260811 22:19:48.674818 15093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:48.759887 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 14889
I20260811 22:19:48.798393 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 15105
I20260811 22:19:48.807384 15350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:48.799598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51844 (local address 127.29.229.3:41715)
0811 22:19:48.800287 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:48.800294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:48.800326 (+ 32us) server_negotiation.cc:407] Connection header received
0811 22:19:48.800388 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:48.800394 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:48.800698 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:48.800835 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:48.801289 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:48.802396 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:48.803747 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:48.804434 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:48.804601 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:48.805870 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:48.805880 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:48.805904 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:19:48.806491 (+ 587us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:48.806539 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:19:48.806559 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:48.806666 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:48.806812 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 22:19:48.806934 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":391,"threads_started":1}
I20260811 22:19:48.808853 15319 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:48.894758 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 15105
I20260811 22:19:48.922425 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 14487
I20260811 22:19:48.923694 14603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:49.032600 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 14487
2026-08-11T22:19:49Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2792 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T22:19:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:19:49Z Disabled control of system clock
I20260811 22:19:49.138720 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:39143
--webserver_interface=127.29.229.62
--webserver_port=0
--builtin_ntp_servers=127.29.229.20:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:39143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 22:19:49.501523 15357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:49.501849 15357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:49.501946 15357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:49.509549 15357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:19:49.509678 15357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:49.509718 15357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:49.509769 15357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:19:49.509893 15357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:19:49.519698 15357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:39143
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:39143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:49.521224 15357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:49.523124 15357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:49.535728 15365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:49.535667 15369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:49.535640 15367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:49.536831 15357 server_base.cc:1034] running on GCE node
I20260811 22:19:49.537735 15357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:49.539315 15357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:49.540601 15357 hybrid_clock.cc:648] HybridClock initialized: now 1786486789540557 us; error 38 us; skew 500 ppm
I20260811 22:19:49.541005 15357 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:19:49.550220 15357 webserver.cc:460] Webserver started at http://127.29.229.62:36715/ using document root <none> and password file <none>
I20260811 22:19:49.551090 15357 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:49.551199 15357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:49.551592 15357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:49.554770 15357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "a90870bfee2a42a1891594225495c8bf"
format_stamp: "Formatted at 2026-08-11 22:19:49 on dist-test-slave-jq4t"
server_key: "56d71068b699e37228442edff1bfddf9"
server_key_iv: "79add6e129762c786820e3a68bb891f2"
server_key_version: "encryptionkey@0"
I20260811 22:19:49.555645 15357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "a90870bfee2a42a1891594225495c8bf"
format_stamp: "Formatted at 2026-08-11 22:19:49 on dist-test-slave-jq4t"
server_key: "56d71068b699e37228442edff1bfddf9"
server_key_iv: "79add6e129762c786820e3a68bb891f2"
server_key_version: "encryptionkey@0"
I20260811 22:19:49.562503 15357 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260811 22:19:49.566601 15377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:49.568188 15357 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:19:49.568409 15357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "a90870bfee2a42a1891594225495c8bf"
format_stamp: "Formatted at 2026-08-11 22:19:49 on dist-test-slave-jq4t"
server_key: "56d71068b699e37228442edff1bfddf9"
server_key_iv: "79add6e129762c786820e3a68bb891f2"
server_key_version: "encryptionkey@0"
I20260811 22:19:49.568738 15357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:49.576050 15357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:49.576993 15357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:49.577322 15357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:49.599685 15357 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:39143
I20260811 22:19:49.599704 15456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:39143 every 8 connection(s)
I20260811 22:19:49.601199 15357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:19:49.605363 15457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:49.611186 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 15357
I20260811 22:19:49.611419 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:19:49.612270 30612 external_mini_cluster.cc:1428] Setting key 7cfd3a429cb3c958026e04f5db95f7d3
I20260811 22:19:49.617949 15457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf: Bootstrap starting.
I20260811 22:19:49.621394 15462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:49.614670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37454 (local address 127.29.229.62:39143)
0811 22:19:49.615919 (+ 1249us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:49.615931 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:49.615980 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:19:49.617723 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:49.617746 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:49.618220 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:49.618628 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:49.619738 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:49.619745 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:49.619756 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:19:49.619771 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:49.619840 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:19:49.619857 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:49.620095 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:49.620232 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 22:19:49.620516 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":431,"threads_started":1}
I20260811 22:19:49.621585 15457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:49.622740 15457 log.cc:824] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:49.625730 15457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf: No bootstrap required, opened a new log
I20260811 22:19:49.633477 15457 raft_consensus.cc:348] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a90870bfee2a42a1891594225495c8bf" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 39143 } }
I20260811 22:19:49.633908 15457 raft_consensus.cc:374] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:49.634022 15457 raft_consensus.cc:729] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a90870bfee2a42a1891594225495c8bf, State: Initialized, Role: FOLLOWER
I20260811 22:19:49.634644 15457 consensus_queue.cc:260] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a90870bfee2a42a1891594225495c8bf" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 39143 } }
I20260811 22:19:49.634878 15457 raft_consensus.cc:388] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:19:49.635010 15457 raft_consensus.cc:482] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:19:49.635205 15457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:49.637593 15457 raft_consensus.cc:504] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a90870bfee2a42a1891594225495c8bf" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 39143 } }
I20260811 22:19:49.638136 15457 leader_election.cc:302] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a90870bfee2a42a1891594225495c8bf; no voters:
I20260811 22:19:49.638829 15457 leader_election.cc:288] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:19:49.639019 15468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:19:49.639794 15468 raft_consensus.cc:686] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf [term 1 LEADER]: Becoming Leader. State: Replica: a90870bfee2a42a1891594225495c8bf, State: Running, Role: LEADER
I20260811 22:19:49.640519 15468 consensus_queue.cc:237] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a90870bfee2a42a1891594225495c8bf" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 39143 } }
I20260811 22:19:49.642131 15457 sys_catalog.cc:564] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:19:49.644445 15471 sys_catalog.cc:455] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader a90870bfee2a42a1891594225495c8bf. Latest consensus state: current_term: 1 leader_uuid: "a90870bfee2a42a1891594225495c8bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a90870bfee2a42a1891594225495c8bf" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 39143 } } }
I20260811 22:19:49.644905 15471 sys_catalog.cc:458] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:49.644572 15469 sys_catalog.cc:455] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a90870bfee2a42a1891594225495c8bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a90870bfee2a42a1891594225495c8bf" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 39143 } } }
I20260811 22:19:49.645045 15469 sys_catalog.cc:458] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:49.646703 15474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:19:49.653074 15474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:19:49.663333 15474 catalog_manager.cc:1347] Generated new cluster ID: 1db36ec67af24b5887ad03ee1fb8d07b
I20260811 22:19:49.663476 15474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:19:49.677474 15474 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:19:49.678081 15474 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:19:49.690948 15474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a90870bfee2a42a1891594225495c8bf: Generated new TSK 0
I20260811 22:19:49.691836 15474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:19:49.706234 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:39143
--builtin_ntp_servers=127.29.229.20:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 22:19:50.076649 15494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:50.076965 15494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:50.077116 15494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:50.084690 15494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:50.084807 15494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:50.084975 15494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:19:50.094718 15494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:39143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:50.096138 15494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:50.098035 15494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:50.112126 15502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:50.112125 15503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:50.112339 15506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:50.112900 15494 server_base.cc:1034] running on GCE node
I20260811 22:19:50.113657 15494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:50.114691 15494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:50.115942 15494 hybrid_clock.cc:648] HybridClock initialized: now 1786486790115883 us; error 43 us; skew 500 ppm
I20260811 22:19:50.116333 15494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:19:50.118801 15494 webserver.cc:460] Webserver started at http://127.29.229.1:40731/ using document root <none> and password file <none>
I20260811 22:19:50.119554 15494 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:50.119673 15494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:50.120036 15494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:50.123312 15494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "5a1a60a79f1646f38454532dd450d686"
format_stamp: "Formatted at 2026-08-11 22:19:50 on dist-test-slave-jq4t"
server_key: "ff4e1615d594df0d55f9252334eb8504"
server_key_iv: "5cc04c1abbcfaae1d030939a74b9905f"
server_key_version: "encryptionkey@0"
I20260811 22:19:50.124147 15494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "5a1a60a79f1646f38454532dd450d686"
format_stamp: "Formatted at 2026-08-11 22:19:50 on dist-test-slave-jq4t"
server_key: "ff4e1615d594df0d55f9252334eb8504"
server_key_iv: "5cc04c1abbcfaae1d030939a74b9905f"
server_key_version: "encryptionkey@0"
I20260811 22:19:50.129894 15494 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 22:19:50.133636 15516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:50.135107 15494 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 22:19:50.135308 15494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "5a1a60a79f1646f38454532dd450d686"
format_stamp: "Formatted at 2026-08-11 22:19:50 on dist-test-slave-jq4t"
server_key: "ff4e1615d594df0d55f9252334eb8504"
server_key_iv: "5cc04c1abbcfaae1d030939a74b9905f"
server_key_version: "encryptionkey@0"
I20260811 22:19:50.135522 15494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:50.141884 15494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:50.142759 15494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:50.143127 15494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:50.144178 15494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:50.145790 15494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:50.145921 15494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:50.146042 15494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:50.146198 15494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:50.183620 15494 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:40005
I20260811 22:19:50.183732 15689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:40005 every 8 connection(s)
I20260811 22:19:50.185250 15494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:19:50.192356 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 15494
I20260811 22:19:50.192655 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:19:50.193518 30612 external_mini_cluster.cc:1428] Setting key d5643c3fffbef5277fd30f091ec1af2e
I20260811 22:19:50.196744 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:39143
--builtin_ntp_servers=127.29.229.20:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 22:19:50.199107 15695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:50.188999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:42801 (local address 127.29.229.62:39143)
0811 22:19:50.189639 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:50.189649 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:50.193706 (+ 4057us) server_negotiation.cc:407] Connection header received
0811 22:19:50.195728 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:50.195733 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:50.195798 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:50.195919 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:50.197881 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:50.197891 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:50.197911 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:19:50.197940 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:50.198006 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:19:50.198029 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:50.198151 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:50.198594 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 22:19:50.198674 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":275,"threads_started":1}
I20260811 22:19:50.201550 15698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:50.189789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:39143 (local address 127.29.229.1:42801)
0811 22:19:50.193487 (+ 3698us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:50.193543 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:50.195459 (+ 1916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:50.196356 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:50.196370 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:50.196521 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:50.197371 (+ 850us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:19:50.197377 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:50.197457 (+ 80us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:19:50.197629 (+ 172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:50.198276 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:50.198291 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:50.198302 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 22:19:50.199495 (+ 1193us) client_negotiation.cc:240] Negotiation successful
0811 22:19:50.199847 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2979,"thread_start_us":2733,"threads_started":1}
I20260811 22:19:50.203646 15690 heartbeater.cc:347] Connected to a master server at 127.29.229.62:39143
I20260811 22:19:50.203953 15690 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:50.204720 15690 heartbeater.cc:511] Master 127.29.229.62:39143 requested a full tablet report, sending...
I20260811 22:19:50.207558 15398 ts_manager.cc:194] Registered new tserver with Master: 5a1a60a79f1646f38454532dd450d686 (127.29.229.1:40005)
W20260811 22:19:50.558657 15700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:50.558962 15700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:50.559121 15700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:50.566535 15700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:50.566644 15700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:50.566821 15700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:19:50.576297 15700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:39143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:50.577636 15700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:50.579396 15700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:50.594316 15706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:50.594275 15711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:50.594362 15707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:50.596165 15700 server_base.cc:1034] running on GCE node
I20260811 22:19:50.596982 15700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:50.598317 15700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:50.599577 15700 hybrid_clock.cc:648] HybridClock initialized: now 1786486790599468 us; error 99 us; skew 500 ppm
I20260811 22:19:50.599946 15700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:19:50.602907 15700 webserver.cc:460] Webserver started at http://127.29.229.2:38413/ using document root <none> and password file <none>
I20260811 22:19:50.603771 15700 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:50.603873 15700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:50.604212 15700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:50.608323 15700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "8ce5b0972e8b485e9783e62589a85163"
format_stamp: "Formatted at 2026-08-11 22:19:50 on dist-test-slave-jq4t"
server_key: "17751b30e9784ce822d7eced91b30cdd"
server_key_iv: "8a8c8bace587674ca97182e60a2bc16e"
server_key_version: "encryptionkey@0"
I20260811 22:19:50.609377 15700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "8ce5b0972e8b485e9783e62589a85163"
format_stamp: "Formatted at 2026-08-11 22:19:50 on dist-test-slave-jq4t"
server_key: "17751b30e9784ce822d7eced91b30cdd"
server_key_iv: "8a8c8bace587674ca97182e60a2bc16e"
server_key_version: "encryptionkey@0"
I20260811 22:19:50.615684 15700 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 22:19:50.619380 15724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:50.620860 15700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:19:50.621042 15700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "8ce5b0972e8b485e9783e62589a85163"
format_stamp: "Formatted at 2026-08-11 22:19:50 on dist-test-slave-jq4t"
server_key: "17751b30e9784ce822d7eced91b30cdd"
server_key_iv: "8a8c8bace587674ca97182e60a2bc16e"
server_key_version: "encryptionkey@0"
I20260811 22:19:50.621279 15700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:50.627159 15700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:50.627954 15700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:50.628242 15700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:50.629238 15700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:50.630780 15700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:50.630923 15700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:50.631049 15700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:50.631139 15700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:50.667340 15700 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:39829
I20260811 22:19:50.667402 15923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:39829 every 8 connection(s)
I20260811 22:19:50.668771 15700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:19:50.671599 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 15700
I20260811 22:19:50.671801 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:19:50.672344 30612 external_mini_cluster.cc:1428] Setting key 3d5f311ac35266c208fdc6c7bb9926f7
I20260811 22:19:50.675338 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:39143
--builtin_ntp_servers=127.29.229.20:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 22:19:50.678162 15695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:50.671508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:53589 (local address 127.29.229.62:39143)
0811 22:19:50.671694 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:50.671702 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:50.673548 (+ 1846us) server_negotiation.cc:407] Connection header received
0811 22:19:50.675479 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:50.675484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:50.675565 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:50.675659 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:50.677320 (+ 1661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:50.677326 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:50.677329 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:19:50.677346 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:50.677415 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:19:50.677434 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:50.677567 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:50.677818 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 22:19:50.677889 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:19:50.679138 15926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:50.672178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:39143 (local address 127.29.229.2:53589)
0811 22:19:50.673405 (+ 1227us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:50.673444 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:50.675206 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:50.675938 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:50.675949 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:50.676087 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:50.676958 (+ 871us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:19:50.676962 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:50.677012 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:19:50.677128 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:50.677618 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:50.677628 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:50.677634 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:19:50.677919 (+ 285us) client_negotiation.cc:240] Negotiation successful
0811 22:19:50.678177 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":409,"threads_started":1}
I20260811 22:19:50.680828 15924 heartbeater.cc:347] Connected to a master server at 127.29.229.62:39143
I20260811 22:19:50.681157 15924 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:50.681747 15924 heartbeater.cc:511] Master 127.29.229.62:39143 requested a full tablet report, sending...
I20260811 22:19:50.683313 15398 ts_manager.cc:194] Registered new tserver with Master: 8ce5b0972e8b485e9783e62589a85163 (127.29.229.2:39829)
W20260811 22:19:51.039373 15928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:51.039722 15928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:51.039898 15928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:51.047574 15928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:51.047730 15928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:51.047920 15928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:19:51.057591 15928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:39143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:51.059796 15928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:51.062179 15928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:51.075899 15939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:51.075949 15935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:51.076040 15934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:51.076715 15928 server_base.cc:1034] running on GCE node
I20260811 22:19:51.077337 15928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:51.078416 15928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:51.079681 15928 hybrid_clock.cc:648] HybridClock initialized: now 1786486791079621 us; error 46 us; skew 500 ppm
I20260811 22:19:51.080052 15928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:19:51.082954 15928 webserver.cc:460] Webserver started at http://127.29.229.3:41697/ using document root <none> and password file <none>
I20260811 22:19:51.083684 15928 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:51.083817 15928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:51.084156 15928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:51.087260 15928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "3be978c9bec049d7a455d1529226b2a2"
format_stamp: "Formatted at 2026-08-11 22:19:51 on dist-test-slave-jq4t"
server_key: "dcb78c060e78bb57d0b79c8d97ebb110"
server_key_iv: "c739b9e7f57fab442f21de0fff08ba83"
server_key_version: "encryptionkey@0"
I20260811 22:19:51.088109 15928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "3be978c9bec049d7a455d1529226b2a2"
format_stamp: "Formatted at 2026-08-11 22:19:51 on dist-test-slave-jq4t"
server_key: "dcb78c060e78bb57d0b79c8d97ebb110"
server_key_iv: "c739b9e7f57fab442f21de0fff08ba83"
server_key_version: "encryptionkey@0"
I20260811 22:19:51.094070 15928 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 22:19:51.097931 15952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:51.099390 15928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:19:51.099591 15928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "3be978c9bec049d7a455d1529226b2a2"
format_stamp: "Formatted at 2026-08-11 22:19:51 on dist-test-slave-jq4t"
server_key: "dcb78c060e78bb57d0b79c8d97ebb110"
server_key_iv: "c739b9e7f57fab442f21de0fff08ba83"
server_key_version: "encryptionkey@0"
I20260811 22:19:51.099830 15928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:51.106009 15928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:51.106841 15928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:51.107138 15928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:51.108237 15928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:51.110040 15928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:51.110183 15928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:51.110317 15928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:51.110415 15928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:51.148524 15928 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:37307
I20260811 22:19:51.148636 16139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:37307 every 8 connection(s)
I20260811 22:19:51.149986 15928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:19:51.150954 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 15928
I20260811 22:19:51.151163 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:19:51.151849 30612 external_mini_cluster.cc:1428] Setting key f69da62c2452917dfa9db6a7bdc19b3a
I20260811 22:19:51.159518 15695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:51.153595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:58285 (local address 127.29.229.62:39143)
0811 22:19:51.153766 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:51.153770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:51.155526 (+ 1756us) server_negotiation.cc:407] Connection header received
0811 22:19:51.157336 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:51.157340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:51.157395 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:51.157490 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:51.158890 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:51.158895 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:51.158898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:19:51.158916 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:51.158949 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:19:51.158961 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:51.159067 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:51.159237 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 22:19:51.159288 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:19:51.160399 16144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:51.154257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:39143 (local address 127.29.229.3:58285)
0811 22:19:51.155360 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:51.155413 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:51.157061 (+ 1648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:51.157740 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:51.157748 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:51.157851 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:51.158575 (+ 724us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:19:51.158578 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:51.158616 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:19:51.158703 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:51.159056 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:51.159065 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:51.159070 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:19:51.159285 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 22:19:51.159471 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":356,"threads_started":1}
I20260811 22:19:51.161809 16140 heartbeater.cc:347] Connected to a master server at 127.29.229.62:39143
I20260811 22:19:51.162147 16140 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:51.162784 16140 heartbeater.cc:511] Master 127.29.229.62:39143 requested a full tablet report, sending...
I20260811 22:19:51.164292 15398 ts_manager.cc:194] Registered new tserver with Master: 3be978c9bec049d7a455d1529226b2a2 (127.29.229.3:37307)
I20260811 22:19:51.167390 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:19:51.174616 15695 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:19:51.172705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37468 (local address 127.29.229.62:39143)
0811 22:19:51.173016 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:51.173025 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:51.173542 (+ 517us) server_negotiation.cc:407] Connection header received
0811 22:19:51.173747 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:51.173753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:51.173823 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:51.173931 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:51.174299 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37468: BlockingRecv error: recv got EOF from 127.0.0.1:37468 (error 108)
Metrics: {"server-negotiator.queue_time_us":145}
I20260811 22:19:51.175143 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 15494
I20260811 22:19:51.179958 16154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:51.176918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41388 (local address 127.29.229.1:40005)
0811 22:19:51.177573 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:51.177581 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:51.177608 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:19:51.177669 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:51.177675 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:51.177924 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:51.178098 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:51.179174 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:51.179181 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:51.179193 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:19:51.179214 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:51.179265 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:19:51.179279 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:51.179435 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:51.179556 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 22:19:51.179663 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":336,"threads_started":1}
I20260811 22:19:51.181357 15684 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:51.267671 15690 heartbeater.cc:503] Master 127.29.229.62:39143 was elected leader, sending a full tablet report...
I20260811 22:19:51.269189 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 15494
W20260811 22:19:51.294960 15382 connection.cc:535] server connection from 127.29.229.1:42801 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:19:51.295388 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 15700
I20260811 22:19:51.300879 16156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:51.297121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40506 (local address 127.29.229.2:39829)
0811 22:19:51.298216 (+ 1095us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:51.298232 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:51.298305 (+ 73us) server_negotiation.cc:407] Connection header received
0811 22:19:51.298452 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:51.298469 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:51.298945 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:51.299115 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:51.300110 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:51.300117 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:51.300129 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:19:51.300143 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:51.300188 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:19:51.300203 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:51.300311 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:51.300398 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 22:19:51.300537 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":403,"threads_started":1}
I20260811 22:19:51.302197 15916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:51.390129 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 15700
I20260811 22:19:51.415657 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 15928
I20260811 22:19:51.419695 16159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:51.416744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40406 (local address 127.29.229.3:37307)
0811 22:19:51.417488 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:51.417495 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:51.417528 (+ 33us) server_negotiation.cc:407] Connection header received
0811 22:19:51.417595 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:51.417601 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:51.417850 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:51.417989 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:51.418909 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:51.418915 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:51.418926 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:19:51.418944 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:51.418985 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:19:51.418999 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:51.419231 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:51.419322 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 22:19:51.419415 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":406,"threads_started":1}
I20260811 22:19:51.421093 16132 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:51.506727 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 15928
I20260811 22:19:51.536718 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 15357
I20260811 22:19:51.537985 15448 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:51.647928 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 15357
2026-08-11T22:19:51Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2608 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:19:51 dist-test-slave-jq4t krb5kdc[16169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:19:51 dist-test-slave-jq4t krb5kdc[16169](info): set up 2 sockets
Aug 11 22:19:51 dist-test-slave-jq4t krb5kdc[16169](info): commencing operation
krb5kdc: starting...
W20260811 22:19:53.032716 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.336s user 0.006s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:19:53 dist-test-slave-jq4t krb5kdc[16169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:19:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:19:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:19:53.377341 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:38439
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:43187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:38439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:53.752640 16188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:53.752943 16188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:53.753036 16188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:53.760280 16188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:19:53.760404 16188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:53.760452 16188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:53.760584 16188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:19:53.760637 16188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:19:53.770872 16188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:43187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:38439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:38439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:53.772225 16188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:53.773749 16188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:53.786324 16194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:53.786300 16195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:53.786278 16197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:53.786669 16188 server_base.cc:1034] running on GCE node
I20260811 22:19:53.787861 16188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:53.789505 16188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:53.790763 16188 hybrid_clock.cc:648] HybridClock initialized: now 1786486793790699 us; error 57 us; skew 500 ppm
I20260811 22:19:53.791169 16188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:53 dist-test-slave-jq4t krb5kdc[16169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486793, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:53.794634 16188 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:19:53.796409 16188 webserver.cc:460] Webserver started at http://127.29.229.62:36793/ using document root <none> and password file <none>
I20260811 22:19:53.797161 16188 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:53.797268 16188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:53.797538 16188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:53.800645 16188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "50aba45cd23449d38b53a7422ebae551"
format_stamp: "Formatted at 2026-08-11 22:19:53 on dist-test-slave-jq4t"
server_key: "9090e227c397d04eb3961da6e9cd997d"
server_key_iv: "5e947dc5e2e55b4eaeb1a1ca0561e970"
server_key_version: "encryptionkey@0"
I20260811 22:19:53.801398 16188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "50aba45cd23449d38b53a7422ebae551"
format_stamp: "Formatted at 2026-08-11 22:19:53 on dist-test-slave-jq4t"
server_key: "9090e227c397d04eb3961da6e9cd997d"
server_key_iv: "5e947dc5e2e55b4eaeb1a1ca0561e970"
server_key_version: "encryptionkey@0"
I20260811 22:19:53.806977 16188 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:19:53.810710 16207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:53.812180 16188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:19:53.812357 16188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "50aba45cd23449d38b53a7422ebae551"
format_stamp: "Formatted at 2026-08-11 22:19:53 on dist-test-slave-jq4t"
server_key: "9090e227c397d04eb3961da6e9cd997d"
server_key_iv: "5e947dc5e2e55b4eaeb1a1ca0561e970"
server_key_version: "encryptionkey@0"
I20260811 22:19:53.813017 16188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:53.844063 16188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:53.848672 16188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:53.848941 16188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:53.868646 16188 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:38439
I20260811 22:19:53.868666 16293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:38439 every 8 connection(s)
I20260811 22:19:53.870121 16188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:19:53.874193 16296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:53.875398 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 16188
I20260811 22:19:53.875670 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:19:53.876595 30612 external_mini_cluster.cc:1428] Setting key babac80de9bdfa6499bc378cc3e7b357
I20260811 22:19:53.889856 16296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551: Bootstrap starting.
Aug 11 22:19:53 dist-test-slave-jq4t krb5kdc[16169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:19:53.893986 16296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:53.895339 16296 log.cc:824] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:53.898985 16296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551: No bootstrap required, opened a new log
I20260811 22:19:53.902987 16300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:53.878880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52168 (local address 127.29.229.62:38439)
0811 22:19:53.879743 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:53.879754 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:53.879799 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:19:53.881487 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:53.881512 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:53.881909 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:53.882254 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:53.884155 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:53.885538 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:53.887422 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:53.888223 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:53.888385 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:53.893385 (+ 5000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:53.893396 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:53.893415 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:53.893483 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:53.897928 (+ 4445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:53.898714 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:53.898722 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:53.898732 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:53.898948 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:53.899415 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:53.899423 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:53.899436 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:53.900260 (+ 824us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:53.900616 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:53.901063 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 22:19:53.901542 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":432,"threads_started":1}
I20260811 22:19:53.909261 16296 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50aba45cd23449d38b53a7422ebae551" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 38439 } }
I20260811 22:19:53.909641 16296 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:53.909715 16296 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50aba45cd23449d38b53a7422ebae551, State: Initialized, Role: FOLLOWER
I20260811 22:19:53.910300 16296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50aba45cd23449d38b53a7422ebae551" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 38439 } }
I20260811 22:19:53.910512 16296 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:19:53.910612 16296 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:19:53.910743 16296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:53.913014 16296 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50aba45cd23449d38b53a7422ebae551" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 38439 } }
I20260811 22:19:53.913466 16296 leader_election.cc:302] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50aba45cd23449d38b53a7422ebae551; no voters:
I20260811 22:19:53.914125 16296 leader_election.cc:288] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:19:53.914388 16303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:19:53.915485 16303 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551 [term 1 LEADER]: Becoming Leader. State: Replica: 50aba45cd23449d38b53a7422ebae551, State: Running, Role: LEADER
I20260811 22:19:53.916107 16303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50aba45cd23449d38b53a7422ebae551" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 38439 } }
I20260811 22:19:53.916543 16296 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:19:53.919202 16310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50aba45cd23449d38b53a7422ebae551. Latest consensus state: current_term: 1 leader_uuid: "50aba45cd23449d38b53a7422ebae551" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50aba45cd23449d38b53a7422ebae551" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 38439 } } }
I20260811 22:19:53.919667 16310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:53.920192 16305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50aba45cd23449d38b53a7422ebae551" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50aba45cd23449d38b53a7422ebae551" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 38439 } } }
I20260811 22:19:53.920589 16305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551 [sys.catalog]: This master's current role is: LEADER
I20260811 22:19:53.921478 16314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:19:53.928263 16314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:19:53.938304 16314 catalog_manager.cc:1347] Generated new cluster ID: a66f22d6390746c08abfc235b1362c07
I20260811 22:19:53.938453 16314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:19:53.969471 16314 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:19:53.971110 16314 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:19:53.985024 16314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551: Generated new TSK 0
I20260811 22:19:53.986239 16314 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
I20260811 22:19:54.088104 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:38439
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:43187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:54.482756 16343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:54.483072 16343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:54.483186 16343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:54.491568 16343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:54.491703 16343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:54.491822 16343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:19:54.502421 16343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:43187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:38439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:54.503952 16343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:54.505290 16343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:54.519922 16351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:54.519896 16352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:54.520457 16354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:54.521202 16343 server_base.cc:1034] running on GCE node
I20260811 22:19:54.522169 16343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:54.523486 16343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:54.524993 16343 hybrid_clock.cc:648] HybridClock initialized: now 1786486794524920 us; error 56 us; skew 500 ppm
I20260811 22:19:54.525432 16343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:54 dist-test-slave-jq4t krb5kdc[16169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486794, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:54.529135 16343 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:19:54.530934 16343 webserver.cc:460] Webserver started at http://127.29.229.1:45637/ using document root <none> and password file <none>
I20260811 22:19:54.532267 16343 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:54.532418 16343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:54.532811 16343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:54.536043 16343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "de7f64630ee443b49087eba797b055d7"
format_stamp: "Formatted at 2026-08-11 22:19:54 on dist-test-slave-jq4t"
server_key: "c5d3e14bd6b180bd97e4855e2141719c"
server_key_iv: "9019dcc1c52ed64e62257081cfd643f8"
server_key_version: "encryptionkey@0"
I20260811 22:19:54.536923 16343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "de7f64630ee443b49087eba797b055d7"
format_stamp: "Formatted at 2026-08-11 22:19:54 on dist-test-slave-jq4t"
server_key: "c5d3e14bd6b180bd97e4855e2141719c"
server_key_iv: "9019dcc1c52ed64e62257081cfd643f8"
server_key_version: "encryptionkey@0"
I20260811 22:19:54.543306 16343 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 22:19:54.547837 16363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:54.549935 16343 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 22:19:54.550287 16343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "de7f64630ee443b49087eba797b055d7"
format_stamp: "Formatted at 2026-08-11 22:19:54 on dist-test-slave-jq4t"
server_key: "c5d3e14bd6b180bd97e4855e2141719c"
server_key_iv: "9019dcc1c52ed64e62257081cfd643f8"
server_key_version: "encryptionkey@0"
I20260811 22:19:54.550547 16343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:54.579941 16343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:54.585542 16343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:54.585870 16343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:54.586951 16343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:54.588650 16343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:54.588776 16343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:54.588954 16343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:54.589037 16343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:54.632205 16343 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:34503
I20260811 22:19:54.632266 16549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:34503 every 8 connection(s)
I20260811 22:19:54.633637 16343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:19:54.642316 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 16343
I20260811 22:19:54.642593 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:19:54.643276 30612 external_mini_cluster.cc:1428] Setting key eff9cb61fc9baa97bdceaf740b6b5bb6
Aug 11 22:19:54 dist-test-slave-jq4t krb5kdc[16169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486794, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:19:54.659060 16555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:54.636434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:36867 (local address 127.29.229.62:38439)
0811 22:19:54.636974 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:54.636978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:54.637826 (+ 848us) server_negotiation.cc:407] Connection header received
0811 22:19:54.639772 (+ 1946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:54.639777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:54.639881 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:54.640016 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:54.641788 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:54.642599 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:54.644449 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:54.645454 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:54.645584 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:54.649898 (+ 4314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:54.649909 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:54.649916 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:54.649969 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:54.654430 (+ 4461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:54.655494 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:54.655502 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:54.655507 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:54.655682 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:54.656691 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:54.656699 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:54.656705 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:54.657152 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:54.657332 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:54.658262 (+ 930us) server_negotiation.cc:299] Negotiation successful
0811 22:19:54.658552 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":269,"threads_started":1}
I20260811 22:19:54.660027 16559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:54.636893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:38439 (local address 127.29.229.1:36867)
0811 22:19:54.637647 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:54.637689 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:54.639487 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:54.640322 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:54.640331 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:54.640989 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:54.641593 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:54.641616 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:54.642771 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:54.642776 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:54.644143 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:54.644157 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:54.645746 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:54.645753 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:54.646116 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:54.646984 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:54.647009 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:54.649411 (+ 2402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:54.654660 (+ 5249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:54.654669 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:54.654684 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:54.655339 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:54.655933 (+ 594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:54.655939 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:54.655942 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:54.656435 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:54.657356 (+ 921us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:54.657363 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:54.657798 (+ 435us) client_negotiation.cc:769] Sending connection context
0811 22:19:54.658092 (+ 294us) client_negotiation.cc:240] Negotiation successful
0811 22:19:54.658678 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":299,"threads_started":1}
I20260811 22:19:54.661846 16551 heartbeater.cc:347] Connected to a master server at 127.29.229.62:38439
I20260811 22:19:54.662227 16551 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:54.663283 16551 heartbeater.cc:511] Master 127.29.229.62:38439 requested a full tablet report, sending...
I20260811 22:19:54.666467 16245 ts_manager.cc:194] Registered new tserver with Master: de7f64630ee443b49087eba797b055d7 (127.29.229.1:34503)
I20260811 22:19:54.668452 16245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:36867
WARNING: no policy specified for kudu/127.29.229.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
I20260811 22:19:54.722399 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:38439
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:43187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:19:55.099644 16563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:55.100023 16563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:55.100214 16563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:55.108630 16563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:55.108798 16563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:55.108995 16563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:19:55.119827 16563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:43187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:38439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:55.122021 16563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:55.123481 16563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:55.139142 16572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:55.139132 16576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:55.139132 16571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:55.142016 16563 server_base.cc:1034] running on GCE node
I20260811 22:19:55.142760 16563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:55.144395 16563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:55.145774 16563 hybrid_clock.cc:648] HybridClock initialized: now 1786486795145712 us; error 40 us; skew 500 ppm
I20260811 22:19:55.146209 16563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:55 dist-test-slave-jq4t krb5kdc[16169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486795, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:55.150172 16563 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:19:55.152079 16563 webserver.cc:460] Webserver started at http://127.29.229.2:37207/ using document root <none> and password file <none>
I20260811 22:19:55.152978 16563 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:55.153144 16563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:55.153474 16563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:55.156883 16563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "543036741f714f5ab282a27a0b1aeeab"
format_stamp: "Formatted at 2026-08-11 22:19:55 on dist-test-slave-jq4t"
server_key: "431eda4ea71d9c3a08c4b017784d011b"
server_key_iv: "19b063f121d9962cf010aabad504ee19"
server_key_version: "encryptionkey@0"
I20260811 22:19:55.157712 16563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "543036741f714f5ab282a27a0b1aeeab"
format_stamp: "Formatted at 2026-08-11 22:19:55 on dist-test-slave-jq4t"
server_key: "431eda4ea71d9c3a08c4b017784d011b"
server_key_iv: "19b063f121d9962cf010aabad504ee19"
server_key_version: "encryptionkey@0"
I20260811 22:19:55.163772 16563 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 22:19:55.168035 16584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:55.169751 16563 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:19:55.169956 16563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "543036741f714f5ab282a27a0b1aeeab"
format_stamp: "Formatted at 2026-08-11 22:19:55 on dist-test-slave-jq4t"
server_key: "431eda4ea71d9c3a08c4b017784d011b"
server_key_iv: "19b063f121d9962cf010aabad504ee19"
server_key_version: "encryptionkey@0"
I20260811 22:19:55.170178 16563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:55.197906 16563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:55.202634 16563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:55.202975 16563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:55.204151 16563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:55.206264 16563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:55.206377 16563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:55.206513 16563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:55.206581 16563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:55.259220 16563 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:33279
I20260811 22:19:55.259328 16770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:33279 every 8 connection(s)
I20260811 22:19:55.260591 16563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:19:55.261525 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 16563
I20260811 22:19:55.261761 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:19:55.262326 30612 external_mini_cluster.cc:1428] Setting key 6934f0648d37b61022ee9a3d52672b31
Aug 11 22:19:55 dist-test-slave-jq4t krb5kdc[16169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486795, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:19:55.284799 16774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:55.263766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:36583 (local address 127.29.229.62:38439)
0811 22:19:55.264329 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:55.264335 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:55.265618 (+ 1283us) server_negotiation.cc:407] Connection header received
0811 22:19:55.267515 (+ 1897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:55.267523 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:55.267622 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:55.267745 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:55.269484 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.270634 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.272701 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.273443 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.273608 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:55.277565 (+ 3957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:55.277574 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:55.277582 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:55.277629 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:55.280745 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:55.281768 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:55.281773 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:55.281783 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:55.281959 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:55.282544 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:55.282553 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:55.282555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:55.282906 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:55.283040 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:55.284225 (+ 1185us) server_negotiation.cc:299] Negotiation successful
0811 22:19:55.284415 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":333,"threads_started":1}
I20260811 22:19:55.286260 16777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:55.264570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:38439 (local address 127.29.229.2:36583)
0811 22:19:55.265436 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:55.265483 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:55.267235 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:55.268060 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:55.268069 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:55.268695 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:55.269291 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:55.269309 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.270817 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.270831 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:55.272315 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:55.272348 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.273796 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.273804 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:55.274122 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:55.274913 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:55.274969 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:55.277290 (+ 2321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:55.280917 (+ 3627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:55.280929 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:55.280949 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:55.281624 (+ 675us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:55.282102 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:55.282109 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:55.282115 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:55.282374 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:55.283056 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:55.283068 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:55.283553 (+ 485us) client_negotiation.cc:769] Sending connection context
0811 22:19:55.283887 (+ 334us) client_negotiation.cc:240] Negotiation successful
0811 22:19:55.284542 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":377,"threads_started":1}
I20260811 22:19:55.288149 16771 heartbeater.cc:347] Connected to a master server at 127.29.229.62:38439
I20260811 22:19:55.288580 16771 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:55.289381 16771 heartbeater.cc:511] Master 127.29.229.62:38439 requested a full tablet report, sending...
I20260811 22:19:55.291105 16245 ts_manager.cc:194] Registered new tserver with Master: 543036741f714f5ab282a27a0b1aeeab (127.29.229.2:33279)
I20260811 22:19:55.291989 16245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:36583
WARNING: no policy specified for kudu/127.29.229.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
I20260811 22:19:55.339687 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:38439
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:43187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:19:55.673157 16551 heartbeater.cc:503] Master 127.29.229.62:38439 was elected leader, sending a full tablet report...
W20260811 22:19:55.733760 16781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:19:55.734109 16781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:19:55.734282 16781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:19:55.741886 16781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:19:55.742054 16781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:19:55.742233 16781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:19:55.753047 16781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:43187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:38439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:19:55.754514 16781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:19:55.756052 16781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:19:55.769836 16795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:55.769909 16790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:19:55.769932 16789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:19:55.770841 16781 server_base.cc:1034] running on GCE node
I20260811 22:19:55.771505 16781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:19:55.772590 16781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:19:55.773846 16781 hybrid_clock.cc:648] HybridClock initialized: now 1786486795773781 us; error 56 us; skew 500 ppm
I20260811 22:19:55.774257 16781 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:19:55 dist-test-slave-jq4t krb5kdc[16169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486795, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:19:55.777752 16781 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:19:55.779381 16781 webserver.cc:460] Webserver started at http://127.29.229.3:46495/ using document root <none> and password file <none>
I20260811 22:19:55.780129 16781 fs_manager.cc:359] Metadata directory not provided
I20260811 22:19:55.780287 16781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:19:55.780682 16781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:19:55.783854 16781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "d5e25ee285914a78b5cec4a77c236da4"
format_stamp: "Formatted at 2026-08-11 22:19:55 on dist-test-slave-jq4t"
server_key: "bb02ec3dfdf5e2b66aaee338f1e7ce32"
server_key_iv: "250c9420b0c42d2e57e626b74b0ac43b"
server_key_version: "encryptionkey@0"
I20260811 22:19:55.784781 16781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "d5e25ee285914a78b5cec4a77c236da4"
format_stamp: "Formatted at 2026-08-11 22:19:55 on dist-test-slave-jq4t"
server_key: "bb02ec3dfdf5e2b66aaee338f1e7ce32"
server_key_iv: "250c9420b0c42d2e57e626b74b0ac43b"
server_key_version: "encryptionkey@0"
I20260811 22:19:55.790162 16781 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:19:55.793663 16803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:55.795017 16781 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:19:55.795153 16781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "d5e25ee285914a78b5cec4a77c236da4"
format_stamp: "Formatted at 2026-08-11 22:19:55 on dist-test-slave-jq4t"
server_key: "bb02ec3dfdf5e2b66aaee338f1e7ce32"
server_key_iv: "250c9420b0c42d2e57e626b74b0ac43b"
server_key_version: "encryptionkey@0"
I20260811 22:19:55.795326 16781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:19:55.817775 16781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:19:55.823398 16781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:19:55.823733 16781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:19:55.824954 16781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:19:55.826483 16781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:19:55.826567 16781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:55.826655 16781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:19:55.826700 16781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:19:55.863987 16781 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:44751
I20260811 22:19:55.864045 17001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:44751 every 8 connection(s)
I20260811 22:19:55.865378 16781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:19:55.875414 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 16781
I20260811 22:19:55.875649 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:19:55.876238 30612 external_mini_cluster.cc:1428] Setting key 9128c617d7dfc89c4084c912dbcde418
Aug 11 22:19:55 dist-test-slave-jq4t krb5kdc[16169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486795, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:19:55.889073 17006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:55.868143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:53813 (local address 127.29.229.62:38439)
0811 22:19:55.868676 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:55.868682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:55.869773 (+ 1091us) server_negotiation.cc:407] Connection header received
0811 22:19:55.871743 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:55.871752 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:55.871874 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:55.872007 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:55.873763 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.874952 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.877445 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.878011 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.878113 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:55.882244 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:55.882249 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:55.882252 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:55.882286 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:55.885034 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:55.885823 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:55.885828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:55.885830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:55.885993 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:55.886522 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:55.886525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:55.886528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:55.886820 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:55.886959 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:55.888571 (+ 1612us) server_negotiation.cc:299] Negotiation successful
0811 22:19:55.888763 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":260,"threads_started":1}
I20260811 22:19:55.890709 17007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:55.868740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:38439 (local address 127.29.229.3:53813)
0811 22:19:55.869596 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:55.869649 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:55.871458 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:55.872244 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:55.872253 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:55.872883 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:19:55.873575 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:55.873596 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.875161 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.875167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:55.877266 (+ 2099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:55.877281 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.878256 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.878262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:55.878594 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:55.879435 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:19:55.879463 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:19:55.882000 (+ 2537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:19:55.885185 (+ 3185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:55.885196 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:55.885209 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:55.885666 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:55.886140 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:19:55.886145 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:19:55.886147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:19:55.886370 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:19:55.887267 (+ 897us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:19:55.887298 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:19:55.887908 (+ 610us) client_negotiation.cc:769] Sending connection context
0811 22:19:55.888358 (+ 450us) client_negotiation.cc:240] Negotiation successful
0811 22:19:55.889276 (+ 918us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":370,"threads_started":1}
I20260811 22:19:55.892921 17002 heartbeater.cc:347] Connected to a master server at 127.29.229.62:38439
I20260811 22:19:55.893411 17002 heartbeater.cc:464] Registering TS with master...
I20260811 22:19:55.894755 17002 heartbeater.cc:511] Master 127.29.229.62:38439 requested a full tablet report, sending...
I20260811 22:19:55.896804 16245 ts_manager.cc:194] Registered new tserver with Master: d5e25ee285914a78b5cec4a77c236da4 (127.29.229.3:44751)
I20260811 22:19:55.897693 16245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:53813
I20260811 22:19:55.906296 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:19:55.927284 17006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:55.911719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48870 (local address 127.29.229.62:38439)
0811 22:19:55.911939 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:55.911946 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:55.912581 (+ 635us) server_negotiation.cc:407] Connection header received
0811 22:19:55.912733 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:55.912738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:55.912824 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:55.912934 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:55.914702 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.915458 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.917398 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.918091 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.918202 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:55.920935 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:55.920940 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:55.920944 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:55.920979 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:55.923990 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:55.924816 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:55.924833 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:55.924835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:55.924981 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:55.925497 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:55.925500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:55.925502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:55.925876 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:55.926072 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:55.926596 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 22:19:55.926785 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 22:19:55.931895 16245 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:19:55.938868 16245 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:19:55.977537 17027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:55.953574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43108 (local address 127.29.229.1:34503)
0811 22:19:55.965583 (+ 12009us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:55.965596 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:55.967236 (+ 1640us) server_negotiation.cc:407] Connection header received
0811 22:19:55.967368 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:55.967380 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:55.967724 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:55.967940 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:55.969405 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.971045 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.973918 (+ 2873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.975944 (+ 2026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.976101 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:55.976362 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:55.976988 (+ 626us) server_negotiation.cc:299] Negotiation successful
0811 22:19:55.977176 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11801,"thread_start_us":11623,"threads_started":1}
I20260811 22:19:55.977593 17025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:55.953365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:34503 (local address 127.0.0.1:43108)
0811 22:19:55.966450 (+ 13085us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:55.966664 (+ 214us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:55.966981 (+ 317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:55.968372 (+ 1391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:55.968383 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:55.968428 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:55.969187 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:55.969204 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.971220 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.971225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:55.973716 (+ 2491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:55.973746 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.976397 (+ 2651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.976411 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:55.976677 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:55.976749 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 22:19:55.976906 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 22:19:55.977121 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12429,"mutex_wait_us":1,"thread_start_us":12242,"threads_started":1}
I20260811 22:19:55.978528 17028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:55.953607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:44751 (local address 127.0.0.1:51212)
0811 22:19:55.968968 (+ 15361us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:55.968990 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:55.969250 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:55.970296 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:55.970301 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:55.970328 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:55.971017 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:55.971038 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.973107 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.973115 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:55.975463 (+ 2348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:55.975475 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.977728 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.977735 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:55.977920 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:55.977944 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:19:55.978063 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 22:19:55.978191 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15241,"thread_start_us":15111,"threads_started":1}
I20260811 22:19:55.978665 17031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:55.953944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51212 (local address 127.29.229.3:44751)
0811 22:19:55.968410 (+ 14466us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:55.968417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:55.969197 (+ 780us) server_negotiation.cc:407] Connection header received
0811 22:19:55.969528 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:55.969536 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:55.969799 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:55.970012 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:55.971310 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.972968 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.975628 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.977215 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.977350 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:55.977570 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:55.978121 (+ 551us) server_negotiation.cc:299] Negotiation successful
0811 22:19:55.978286 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14300,"thread_start_us":14131,"threads_started":1}
I20260811 22:19:55.982131 17032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:55.969472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51652 (local address 127.29.229.2:33279)
0811 22:19:55.970130 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:55.970139 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:55.972099 (+ 1960us) server_negotiation.cc:407] Connection header received
0811 22:19:55.972279 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:55.972286 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:55.972547 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:55.972731 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:55.973376 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.975033 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.977455 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.979967 (+ 2512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.980059 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:55.980191 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:55.981573 (+ 1382us) server_negotiation.cc:299] Negotiation successful
0811 22:19:55.981768 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":337,"threads_started":1}
I20260811 22:19:55.981859 16446 tablet_service.cc:1467] Processing CreateTablet for tablet e2a70f6f087748e496314d0120982257 (DEFAULT_TABLE table=test-table [id=cb9eab0ae197499fa562ce34ff4318b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:55.982834 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:55.970291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:33279 (local address 127.0.0.1:51652)
0811 22:19:55.971959 (+ 1668us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:55.971977 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:55.972111 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:55.972741 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:55.972746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:55.972772 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:55.973184 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:55.973196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.975161 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.975166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:55.977239 (+ 2073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:55.977254 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:55.981147 (+ 3893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:55.981153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:55.981351 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:55.981382 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 22:19:55.982355 (+ 973us) client_negotiation.cc:240] Negotiation successful
0811 22:19:55.982514 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1539,"thread_start_us":1321,"threads_started":1}
I20260811 22:19:55.983477 16446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2a70f6f087748e496314d0120982257. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:55.986272 16658 tablet_service.cc:1467] Processing CreateTablet for tablet e2a70f6f087748e496314d0120982257 (DEFAULT_TABLE table=test-table [id=cb9eab0ae197499fa562ce34ff4318b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:55.983656 16882 tablet_service.cc:1467] Processing CreateTablet for tablet e2a70f6f087748e496314d0120982257 (DEFAULT_TABLE table=test-table [id=cb9eab0ae197499fa562ce34ff4318b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:19:55.987521 16658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2a70f6f087748e496314d0120982257. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:55.988610 16882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2a70f6f087748e496314d0120982257. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:19:56.000424 17038 tablet_bootstrap.cc:492] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7: Bootstrap starting.
I20260811 22:19:56.004393 17040 tablet_bootstrap.cc:492] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4: Bootstrap starting.
I20260811 22:19:56.005344 17035 tablet_bootstrap.cc:492] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab: Bootstrap starting.
I20260811 22:19:56.006404 17038 tablet_bootstrap.cc:654] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:56.007966 17038 log.cc:824] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:56.008339 17040 tablet_bootstrap.cc:654] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:56.009476 17035 tablet_bootstrap.cc:654] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab: Neither blocks nor log segments found. Creating new log.
I20260811 22:19:56.009918 17040 log.cc:824] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:56.010695 17035 log.cc:824] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab: Log is configured to *not* fsync() on all Append() calls
I20260811 22:19:56.011795 17038 tablet_bootstrap.cc:492] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7: No bootstrap required, opened a new log
I20260811 22:19:56.012116 17038 ts_tablet_manager.cc:1397] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.008s
I20260811 22:19:56.013216 17040 tablet_bootstrap.cc:492] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4: No bootstrap required, opened a new log
I20260811 22:19:56.013510 17040 ts_tablet_manager.cc:1397] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 22:19:56.020344 17035 tablet_bootstrap.cc:492] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab: No bootstrap required, opened a new log
I20260811 22:19:56.020689 17035 ts_tablet_manager.cc:1397] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.004s
I20260811 22:19:56.023447 17038 raft_consensus.cc:348] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de7f64630ee443b49087eba797b055d7" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34503 } } peers { permanent_uuid: "d5e25ee285914a78b5cec4a77c236da4" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44751 } } peers { permanent_uuid: "543036741f714f5ab282a27a0b1aeeab" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33279 } }
I20260811 22:19:56.023944 17038 raft_consensus.cc:374] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:56.024274 17038 raft_consensus.cc:729] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de7f64630ee443b49087eba797b055d7, State: Initialized, Role: FOLLOWER
I20260811 22:19:56.024546 17040 raft_consensus.cc:348] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de7f64630ee443b49087eba797b055d7" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34503 } } peers { permanent_uuid: "d5e25ee285914a78b5cec4a77c236da4" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44751 } } peers { permanent_uuid: "543036741f714f5ab282a27a0b1aeeab" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33279 } }
I20260811 22:19:56.024993 17040 raft_consensus.cc:374] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:56.025110 17040 raft_consensus.cc:729] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5e25ee285914a78b5cec4a77c236da4, State: Initialized, Role: FOLLOWER
I20260811 22:19:56.025063 17038 consensus_queue.cc:260] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de7f64630ee443b49087eba797b055d7" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34503 } } peers { permanent_uuid: "d5e25ee285914a78b5cec4a77c236da4" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44751 } } peers { permanent_uuid: "543036741f714f5ab282a27a0b1aeeab" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33279 } }
I20260811 22:19:56.025754 17040 consensus_queue.cc:260] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de7f64630ee443b49087eba797b055d7" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34503 } } peers { permanent_uuid: "d5e25ee285914a78b5cec4a77c236da4" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44751 } } peers { permanent_uuid: "543036741f714f5ab282a27a0b1aeeab" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33279 } }
I20260811 22:19:56.027135 17002 heartbeater.cc:503] Master 127.29.229.62:38439 was elected leader, sending a full tablet report...
I20260811 22:19:56.028048 17040 ts_tablet_manager.cc:1428] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260811 22:19:56.029995 17035 raft_consensus.cc:348] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de7f64630ee443b49087eba797b055d7" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34503 } } peers { permanent_uuid: "d5e25ee285914a78b5cec4a77c236da4" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44751 } } peers { permanent_uuid: "543036741f714f5ab282a27a0b1aeeab" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33279 } }
I20260811 22:19:56.030474 17035 raft_consensus.cc:374] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:19:56.030572 17038 ts_tablet_manager.cc:1428] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7: Time spent starting tablet: real 0.018s user 0.006s sys 0.013s
I20260811 22:19:56.030591 17035 raft_consensus.cc:729] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 543036741f714f5ab282a27a0b1aeeab, State: Initialized, Role: FOLLOWER
I20260811 22:19:56.031265 17035 consensus_queue.cc:260] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de7f64630ee443b49087eba797b055d7" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34503 } } peers { permanent_uuid: "d5e25ee285914a78b5cec4a77c236da4" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44751 } } peers { permanent_uuid: "543036741f714f5ab282a27a0b1aeeab" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33279 } }
I20260811 22:19:56.033440 16771 heartbeater.cc:503] Master 127.29.229.62:38439 was elected leader, sending a full tablet report...
I20260811 22:19:56.034328 17035 ts_tablet_manager.cc:1428] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
W20260811 22:19:56.117554 17004 tablet.cc:2369] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:19:56.136650 16552 tablet.cc:2369] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:19:56.160082 17046 raft_consensus.cc:482] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:19:56.160691 17046 raft_consensus.cc:504] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de7f64630ee443b49087eba797b055d7" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34503 } } peers { permanent_uuid: "d5e25ee285914a78b5cec4a77c236da4" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44751 } } peers { permanent_uuid: "543036741f714f5ab282a27a0b1aeeab" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33279 } }
I20260811 22:19:56.162971 17046 leader_election.cc:288] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de7f64630ee443b49087eba797b055d7 (127.29.229.1:34503), d5e25ee285914a78b5cec4a77c236da4 (127.29.229.3:44751)
I20260811 22:19:56.172045 17027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:56.163615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:57549 (local address 127.29.229.1:34503)
0811 22:19:56.163844 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:56.163851 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:56.164936 (+ 1085us) server_negotiation.cc:407] Connection header received
0811 22:19:56.165175 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:56.165181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:56.165254 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:56.165356 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:56.166151 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:56.167372 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:56.169804 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:56.171060 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:56.171184 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:56.171240 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:56.171726 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 22:19:56.171852 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:19:56.172390 17049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:56.163417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:34503 (local address 127.29.229.2:57549)
0811 22:19:56.164815 (+ 1398us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:56.164832 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:56.165019 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:56.165394 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:56.165399 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:56.165427 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:56.165895 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:56.165907 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:56.167626 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:56.167632 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:56.169597 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:56.169613 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:56.171320 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:56.171328 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:56.171512 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:56.171552 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 22:19:56.171670 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 22:19:56.171826 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1271,"mutex_wait_us":33,"thread_start_us":890,"threads_started":1}
I20260811 22:19:56.173482 17031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:56.163660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:50135 (local address 127.29.229.3:44751)
0811 22:19:56.163915 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:56.163921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:56.165806 (+ 1885us) server_negotiation.cc:407] Connection header received
0811 22:19:56.166076 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:56.166084 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:56.166155 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:56.166231 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:19:56.167080 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:56.168210 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:56.170915 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:56.172269 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:56.172470 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:56.172571 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:56.173106 (+ 535us) server_negotiation.cc:299] Negotiation successful
0811 22:19:56.173244 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260811 22:19:56.173614 17050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:56.163512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:44751 (local address 127.29.229.2:50135)
0811 22:19:56.165570 (+ 2058us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:56.165592 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:19:56.165897 (+ 305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:19:56.166383 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:19:56.166390 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:19:56.166425 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:19:56.166871 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:56.166884 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:56.168438 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:56.168451 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:56.170684 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:19:56.170700 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:19:56.172696 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:19:56.172706 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:19:56.172928 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:56.172963 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 22:19:56.173046 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 22:19:56.173182 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1881,"mutex_wait_us":1,"thread_start_us":1597,"threads_started":1}
I20260811 22:19:56.173806 16476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2a70f6f087748e496314d0120982257" candidate_uuid: "543036741f714f5ab282a27a0b1aeeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de7f64630ee443b49087eba797b055d7" is_pre_election: true
I20260811 22:19:56.174482 16476 raft_consensus.cc:2454] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 543036741f714f5ab282a27a0b1aeeab in term 0.
I20260811 22:19:56.174700 16922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2a70f6f087748e496314d0120982257" candidate_uuid: "543036741f714f5ab282a27a0b1aeeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5e25ee285914a78b5cec4a77c236da4" is_pre_election: true
I20260811 22:19:56.175223 16922 raft_consensus.cc:2454] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 543036741f714f5ab282a27a0b1aeeab in term 0.
I20260811 22:19:56.175325 16589 leader_election.cc:302] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 543036741f714f5ab282a27a0b1aeeab, de7f64630ee443b49087eba797b055d7; no voters:
I20260811 22:19:56.175809 17046 raft_consensus.cc:2781] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:19:56.175969 17046 raft_consensus.cc:482] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:19:56.176219 17046 raft_consensus.cc:3037] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:56.178663 17046 raft_consensus.cc:504] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de7f64630ee443b49087eba797b055d7" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34503 } } peers { permanent_uuid: "d5e25ee285914a78b5cec4a77c236da4" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44751 } } peers { permanent_uuid: "543036741f714f5ab282a27a0b1aeeab" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33279 } }
I20260811 22:19:56.179425 17046 leader_election.cc:288] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [CANDIDATE]: Term 1 election: Requested vote from peers de7f64630ee443b49087eba797b055d7 (127.29.229.1:34503), d5e25ee285914a78b5cec4a77c236da4 (127.29.229.3:44751)
I20260811 22:19:56.180188 16476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2a70f6f087748e496314d0120982257" candidate_uuid: "543036741f714f5ab282a27a0b1aeeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de7f64630ee443b49087eba797b055d7"
I20260811 22:19:56.180366 16922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2a70f6f087748e496314d0120982257" candidate_uuid: "543036741f714f5ab282a27a0b1aeeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5e25ee285914a78b5cec4a77c236da4"
I20260811 22:19:56.180454 16476 raft_consensus.cc:3037] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:56.180688 16922 raft_consensus.cc:3037] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:19:56.182863 16476 raft_consensus.cc:2454] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 543036741f714f5ab282a27a0b1aeeab in term 1.
I20260811 22:19:56.182987 16922 raft_consensus.cc:2454] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 543036741f714f5ab282a27a0b1aeeab in term 1.
I20260811 22:19:56.183491 16590 leader_election.cc:302] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 543036741f714f5ab282a27a0b1aeeab, d5e25ee285914a78b5cec4a77c236da4; no voters:
I20260811 22:19:56.183892 17046 raft_consensus.cc:2781] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:19:56.184422 17046 raft_consensus.cc:686] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [term 1 LEADER]: Becoming Leader. State: Replica: 543036741f714f5ab282a27a0b1aeeab, State: Running, Role: LEADER
I20260811 22:19:56.184976 17046 consensus_queue.cc:237] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de7f64630ee443b49087eba797b055d7" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34503 } } peers { permanent_uuid: "d5e25ee285914a78b5cec4a77c236da4" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44751 } } peers { permanent_uuid: "543036741f714f5ab282a27a0b1aeeab" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33279 } }
I20260811 22:19:56.190064 16245 catalog_manager.cc:5495] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab reported cstate change: term changed from 0 to 1, leader changed from <none> to 543036741f714f5ab282a27a0b1aeeab (127.29.229.2). New cstate: current_term: 1 leader_uuid: "543036741f714f5ab282a27a0b1aeeab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de7f64630ee443b49087eba797b055d7" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5e25ee285914a78b5cec4a77c236da4" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "543036741f714f5ab282a27a0b1aeeab" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 33279 } health_report { overall_health: HEALTHY } } }
I20260811 22:19:56.223701 17032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:56.217820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51660 (local address 127.29.229.2:33279)
0811 22:19:56.218058 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:56.218065 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:56.218115 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:19:56.218387 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:56.218391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:56.218489 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:56.218676 (+ 187us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:19:56.219249 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:56.219977 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:56.221422 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:56.221850 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:56.221993 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:56.222498 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:19:56.223054 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:19:56.223224 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:56.223321 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 22:19:56.223411 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 22:19:56.232908 16922 raft_consensus.cc:1261] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4 [term 1 FOLLOWER]: Refusing update from remote peer 543036741f714f5ab282a27a0b1aeeab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:56.232908 16476 raft_consensus.cc:1261] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7 [term 1 FOLLOWER]: Refusing update from remote peer 543036741f714f5ab282a27a0b1aeeab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:19:56.234083 17051 consensus_queue.cc:1035] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5e25ee285914a78b5cec4a77c236da4" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 44751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:56.234513 17046 consensus_queue.cc:1035] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "de7f64630ee443b49087eba797b055d7" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:19:56.245921 17056 mvcc.cc:205] Tried to move back new op lower bound from 7317449917358141440 to 7317449917178474496. Current Snapshot: MvccSnapshot[applied={T|T < 7317449917358141440}]
I20260811 22:19:56.246747 17059 mvcc.cc:205] Tried to move back new op lower bound from 7317449917358141440 to 7317449917178474496. Current Snapshot: MvccSnapshot[applied={T|T < 7317449917358141440}]
I20260811 22:19:56.250658 17058 mvcc.cc:205] Tried to move back new op lower bound from 7317449917358141440 to 7317449917178474496. Current Snapshot: MvccSnapshot[applied={T|T < 7317449917358141440}]
I20260811 22:19:56.262506 16245 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:56.262802 16245 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:19:56.266647 16245 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50aba45cd23449d38b53a7422ebae551: Sending DeleteTablet for 3 replicas of tablet e2a70f6f087748e496314d0120982257
I20260811 22:19:56.267928 16882 tablet_service.cc:1514] Processing DeleteTablet for tablet e2a70f6f087748e496314d0120982257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:56 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:51212
I20260811 22:19:56.267927 16446 tablet_service.cc:1514] Processing DeleteTablet for tablet e2a70f6f087748e496314d0120982257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:56 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:43108
I20260811 22:19:56.270983 16658 tablet_service.cc:1514] Processing DeleteTablet for tablet e2a70f6f087748e496314d0120982257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:19:56 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:51652
I20260811 22:19:56.273861 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 16343
I20260811 22:19:56.291623 17069 tablet_replica.cc:330] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4: stopping tablet replica
I20260811 22:19:56.292178 17069 raft_consensus.cc:2229] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:56.292830 17069 raft_consensus.cc:2258] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:56.294692 17071 tablet_replica.cc:330] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab: stopping tablet replica
I20260811 22:19:56.295343 17069 ts_tablet_manager.cc:1905] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:56.295308 17071 raft_consensus.cc:2229] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:19:56.296190 17071 raft_consensus.cc:2258] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:56.297381 17070 tablet_replica.cc:330] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7: stopping tablet replica
I20260811 22:19:56.297857 17070 raft_consensus.cc:2229] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:19:56.298430 17070 raft_consensus.cc:2258] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:19:56.298928 17071 ts_tablet_manager.cc:1905] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 22:19:56 dist-test-slave-jq4t krb5kdc[16169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:19:56.305011 17069 ts_tablet_manager.cc:1918] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:56.305228 17069 log.cc:1196] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/e2a70f6f087748e496314d0120982257
I20260811 22:19:56.305811 17069 ts_tablet_manager.cc:1939] T e2a70f6f087748e496314d0120982257 P d5e25ee285914a78b5cec4a77c236da4: Deleting consensus metadata
I20260811 22:19:56.309091 16217 catalog_manager.cc:4841] TS d5e25ee285914a78b5cec4a77c236da4 (127.29.229.3:44751): tablet e2a70f6f087748e496314d0120982257 (table test-table [id=cb9eab0ae197499fa562ce34ff4318b6]) successfully deleted
I20260811 22:19:56.309401 17070 ts_tablet_manager.cc:1905] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:19:56.312109 17071 ts_tablet_manager.cc:1918] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:56.312321 17071 log.cc:1196] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/e2a70f6f087748e496314d0120982257
I20260811 22:19:56.312879 17071 ts_tablet_manager.cc:1939] T e2a70f6f087748e496314d0120982257 P 543036741f714f5ab282a27a0b1aeeab: Deleting consensus metadata
I20260811 22:19:56.314360 17027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:56.275652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43120 (local address 127.29.229.1:34503)
0811 22:19:56.275887 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:56.275896 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:56.290692 (+ 14796us) server_negotiation.cc:407] Connection header received
0811 22:19:56.290927 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:56.290940 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:56.291115 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:56.291242 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:56.293964 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:56.295159 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:56.298840 (+ 3681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:56.299505 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:56.299647 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:56.306149 (+ 6502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:56.306161 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:56.306176 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:56.306225 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:56.310766 (+ 4541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:56.311596 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:56.311603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:56.311606 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:56.311767 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:56.312211 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:56.312234 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:56.312240 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:56.312819 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:56.313181 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:56.313551 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 22:19:56.313817 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:19:56.315024 16217 catalog_manager.cc:4841] TS 543036741f714f5ab282a27a0b1aeeab (127.29.229.2:33279): tablet e2a70f6f087748e496314d0120982257 (table test-table [id=cb9eab0ae197499fa562ce34ff4318b6]) successfully deleted
I20260811 22:19:56.315237 16542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:56.412622 17070 ts_tablet_manager.cc:1918] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:19:56.412940 17070 log.cc:1196] T e2a70f6f087748e496314d0120982257 P de7f64630ee443b49087eba797b055d7: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486685119861-30612-0/minicluster-data/ts-0/wal/wals/e2a70f6f087748e496314d0120982257
I20260811 22:19:56.413419 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 16343
I20260811 22:19:56.445134 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 16563
W20260811 22:19:56.445010 16213 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.229.1:34503 (error 108)
I20260811 22:19:56.447247 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:56.446746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:34503 (local address 127.0.0.1:43128)
0811 22:19:56.446934 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:56.447055 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:34503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 22:19:56.447486 16213 catalog_manager.cc:4584] TS de7f64630ee443b49087eba797b055d7 (127.29.229.1:34503): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e2a70f6f087748e496314d0120982257: Network error: Client connection negotiation failed: client connection to 127.29.229.1:34503: connect: Connection refused (error 111)
Aug 11 22:19:56 dist-test-slave-jq4t krb5kdc[16169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.2@KRBTEST.COM
I20260811 22:19:56.465301 17032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:56.446372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51662 (local address 127.29.229.2:33279)
0811 22:19:56.446602 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:56.446609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:56.446760 (+ 151us) server_negotiation.cc:407] Connection header received
0811 22:19:56.446980 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:56.446986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:56.447075 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:56.447201 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:56.449044 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:56.450278 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:56.452263 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:56.452963 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:56.453277 (+ 314us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:56.458788 (+ 5511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:56.458813 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:56.458839 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:56.458886 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:56.462147 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:56.462983 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:56.462991 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:56.462993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:56.463158 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:56.463608 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:56.463612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:56.463614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:56.464057 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:56.464375 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:56.464699 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 22:19:56.464910 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:19:56.466145 16765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:56.501457 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:56.500840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:34503 (local address 127.0.0.1:43130)
0811 22:19:56.501120 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:56.501270 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:34503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 22:19:56.502710 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:56.502314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:34503 (local address 127.0.0.1:43140)
0811 22:19:56.502479 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:56.502566 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:34503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 22:19:56.563549 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:56.562894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:34503 (local address 127.0.0.1:43154)
0811 22:19:56.563197 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:56.563325 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:34503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260811 22:19:56.564747 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:56.564292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:34503 (local address 127.0.0.1:43156)
0811 22:19:56.564504 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:56.564597 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:34503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 22:19:56.594964 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 16563
I20260811 22:19:56.626397 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 16781
Aug 11 22:19:56 dist-test-slave-jq4t krb5kdc[16169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.3@KRBTEST.COM
I20260811 22:19:56.642339 17031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:56.627217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51224 (local address 127.29.229.3:44751)
0811 22:19:56.627435 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 22:19:56.627447 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:19:56.627503 (+ 56us) server_negotiation.cc:407] Connection header received
0811 22:19:56.627730 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:19:56.627736 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:19:56.627830 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:19:56.627966 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:19:56.629359 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:56.630106 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:56.631367 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:19:56.631793 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:19:56.631949 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:19:56.636638 (+ 4689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:19:56.636664 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:19:56.636682 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:19:56.636720 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:19:56.639454 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:56.640145 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:56.640155 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:56.640158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:56.640307 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:19:56.640728 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:19:56.640733 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:19:56.640737 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:19:56.641124 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:19:56.641425 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 22:19:56.641760 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 22:19:56.641992 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 22:19:56.643046 16995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:56.645547 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:56.645090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:34503 (local address 127.0.0.1:43162)
0811 22:19:56.645312 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:56.645400 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:34503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 22:19:56.646696 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:56.646250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:34503 (local address 127.0.0.1:43178)
0811 22:19:56.646460 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:56.646551 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:34503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260811 22:19:56.736399 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 16781
I20260811 22:19:56.764806 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 16188
I20260811 22:19:56.766211 16289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:19:56.881398 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:19:56.880781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:34503 (local address 127.0.0.1:43180)
0811 22:19:56.880988 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 22:19:56.881094 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:34503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 22:19:56.881494 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 16188
2026-08-11T22:19:56Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (5246 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:19:57 dist-test-slave-jq4t krb5kdc[17081](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:19:57 dist-test-slave-jq4t krb5kdc[17081](info): set up 2 sockets
Aug 11 22:19:57 dist-test-slave-jq4t krb5kdc[17081](info): commencing operation
krb5kdc: starting...
W20260811 22:19:58.284056 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.341s user 0.002s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:19:58 dist-test-slave-jq4t krb5kdc[17081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:19:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:19:58Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.229.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.229.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.229.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/rangeradmin_127.29.229.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.229.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/rangeradmin_127.29.229.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.229.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.229.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.229.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/rangerlookup_127.29.229.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.229.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/rangerlookup_127.29.229.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.229.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.229.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.229.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/HTTP_127.29.229.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.229.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/HTTP_127.29.229.20@KRBTEST.COM.keytab.
I20260811 22:19:58.779301 30612 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskD_Yjlj/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/postgres -l logfile start
2026-08-11 22:20:01.597 UTC [17133] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 22:20:01.597 UTC [17133] LOG: listening on IPv4 address "127.29.229.20", port 54545
2026-08-11 22:20:01.601 UTC [17133] LOG: listening on Unix socket "/tmp/.s.PGSQL.54545"
2026-08-11 22:20:01.627 UTC [17139] LOG: database system was shut down at 2026-08-11 22:20:00 UTC
2026-08-11 22:20:01.632 UTC [17133] LOG: database system is ready to accept connections
I20260811 22:20:02.865908 30612 mini_postgres.cc:96] Postgres bound to 54545
2026-08-11 22:20:02.896 UTC [17151] FATAL: database "slave" does not exist
127.29.229.20:54545 - accepting connections
I20260811 22:20:02.897818 30612 mini_ranger.cc:161] Starting Ranger...
I20260811 22:20:02.938369 30612 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 22:20:03.464563 30612 mini_ranger.cc:87] Created ranger Postgres database
I20260811 22:20:03.464730 30612 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ranger-admin
2026-08-11 22:20:03,932 [I] DB FLAVOR :POSTGRES
2026-08-11 22:20:03,933 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:20:03,933 [I] Checking connection..
2026-08-11 22:20:03,933 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 22:20:04,254 [I] Checking connection passed.
2026-08-11 22:20:04,254 [I] --------- Verifying version history table ---------
2026-08-11 22:20:04,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:20:04,537 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:20:04,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:20:04,827 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:20:04,828 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 22:20:04,828 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 22:20:05.084 UTC [17269] WARNING: there is no transaction in progress
2026-08-11 22:20:05,093 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 22:20:05,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:20:05,375 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 22:20:05,376 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:20:05,376 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 22:20:05,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 22:20:05,924 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-jq4t.c.gcp-upstream.internal','N') ;"
2026-08-11 22:20:06,194 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:20:06,485 [I] Table x_portal_user does not exist in database ranger
2026-08-11 22:20:06,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:20:06,758 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 22:20:06,758 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:20:07,040 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:20:07,321 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 22:20:07,322 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 22:20:08.366 UTC [17451] WARNING: there is no transaction in progress
2026-08-11 22:20:08.383 UTC [17451] WARNING: there is no transaction in progress
2026-08-11 22:20:08.401 UTC [17451] WARNING: there is no transaction in progress
2026-08-11 22:20:08.417 UTC [17451] WARNING: there is no transaction in progress
2026-08-11 22:20:08.434 UTC [17451] WARNING: there is no transaction in progress
2026-08-11 22:20:08.455 UTC [17451] WARNING: there is no transaction in progress
2026-08-11 22:20:08.524 UTC [17451] WARNING: there is no transaction in progress
2026-08-11 22:20:08.531 UTC [17451] WARNING: there is no transaction in progress
2026-08-11 22:20:08.545 UTC [17451] WARNING: there is no transaction in progress
2026-08-11 22:20:08.556 UTC [17451] WARNING: there is no transaction in progress
2026-08-11 22:20:08.568 UTC [17451] WARNING: there is no transaction in progress
2026-08-11 22:20:08.583 UTC [17451] WARNING: there is no transaction in progress
2026-08-11 22:20:08.593 UTC [17451] WARNING: there is no transaction in progress
2026-08-11 22:20:08.599 UTC [17451] WARNING: there is no transaction in progress
2026-08-11 22:20:08.607 UTC [17451] WARNING: there is no transaction in progress
2026-08-11 22:20:08.614 UTC [17451] WARNING: there is no transaction in progress
2026-08-11 22:20:09,336 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 22:20:09,336 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 22:20:09,612 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 22:20:09,612 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:20:09,902 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 22:20:09,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:20:10,177 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 22:20:10,177 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:20:10,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:20:10,732 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-jq4t.c.gcp-upstream.internal';"
2026-08-11 22:20:11,000 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 22:20:11,000 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 22:20:11,266 [I] DB_PATCHES have already been applied
I20260811 22:20:11.273345 30612 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ranger-admin:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskD_Yjlj/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 22:20:11.273459 30612 mini_ranger.cc:193] Using host: 127.29.229.20
I20260811 22:20:11.301716 30612 mini_ranger.cc:227] Ranger admin URL: http://127.29.229.20:42161
Aug 11, 2026 10:20:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:20:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:20:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 10:20:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 10:20:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 10:20:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 10:20:12 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 22:20:12.803730 30612 mini_ranger.cc:160] Time spent starting Ranger: real 9.906s user 0.000s sys 0.089s
Aug 11, 2026 10:20:35 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 10:20:36 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 10:20:36 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 10:20:36 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 10:20:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 10:20:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 10:20:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 10:20:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 10:20:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 10:20:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 10:20:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 10:20:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 10:20:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 10:20:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 10:20:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 10:20:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 10:20:36 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 10:20:37 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 22:20:37.749814 30612 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.229.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.229.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.229.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/rangerkms_127.29.229.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.229.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/rangerkms_127.29.229.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.229.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.229.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.229.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/HTTP_127.29.229.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.229.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/HTTP_127.29.229.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 22:20:38 dist-test-slave-jq4t krb5kdc[17081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486838, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 22:20:38.036947 30612 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 22:20:38.073133 30612 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 22:20:39.007157 30612 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 22:20:39.007373 30612 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ranger-kms
I20260811 22:20:39.007507 30612 mini_ranger_kms.cc:227] Using postgres at 127.29.229.20:54545
2026-08-11 22:20:40,016 [I] DB FLAVOR :POSTGRES
2026-08-11 22:20:40,016 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:20:40,016 [I] Checking connection
2026-08-11 22:20:40,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 22:20:40,313 [I] connection success
2026-08-11 22:20:40,313 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 22:20:40,313 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 22:20:40,313 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 22:20:40,603 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 22:20:40,604 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:20:40,604 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 22:20:40,604 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:54545/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 22:20:40,905 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 22:20:41.481969 30612 mini_ranger_kms.cc:310] Created kms service
I20260811 22:20:41.677287 30612 mini_ranger_kms.cc:326] Created kudu user
I20260811 22:20:41.736032 30612 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 22:20:41 dist-test-slave-jq4t krb5kdc[17081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486838, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
I20260811 22:20:42.218989 30612 mini_ranger_kms.cc:384] Added ranger policy
I20260811 22:20:42.220288 30612 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ranger-kms:/tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 22:20:42.220378 30612 mini_ranger_kms.cc:240] Using host: 127.29.229.20
I20260811 22:20:42.256281 30612 mini_ranger_kms.cc:276] Ranger KMS PID: 17730
I20260811 22:20:42.256618 30612 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.229.20:59049
Aug 11, 2026 10:20:43 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:20:43 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:20:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 10:20:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 10:20:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 10:20:43 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 22:20:44 dist-test-slave-jq4t krb5kdc[17081](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486844, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.229.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 22:20:44 dist-test-slave-jq4t krb5kdc[17081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486844, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.229.20@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
I20260811 22:20:45.438870 30612 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 7.402s user 0.153s sys 0.593s
I20260811 22:20:45.439266 30612 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 22:20:45.439483 30612 mini_ranger_kms.cc:401] 127.29.229.20:59049/kms/v1/keys
Aug 11 22:20:45 dist-test-slave-jq4t krb5kdc[17081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486844, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.229.20@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
Aug 11 22:20:48 dist-test-slave-jq4t krb5kdc[17081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:20:48.136317 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:43923
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:43963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:43923
--ranger_config_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.229.20:59049/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:20:48.531536 17810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:48.531940 17810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:48.532119 17810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:48.539904 17810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:20:48.540094 17810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:48.540201 17810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:48.540314 17810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:20:48.540411 17810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:20:48.551295 17810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:43963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.229.20:59049/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:43923
--ranger_config_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:43923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:20:48.552850 17810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:48.554263 17810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:48.567077 17818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:48.567044 17819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:48.566990 17821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:48.568113 17810 server_base.cc:1034] running on GCE node
I20260811 22:20:48.569085 17810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:48.570595 17810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:48.571846 17810 hybrid_clock.cc:648] HybridClock initialized: now 1786486848571720 us; error 108 us; skew 500 ppm
I20260811 22:20:48.572276 17810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:20:48 dist-test-slave-jq4t krb5kdc[17081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486848, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:48.577858 17810 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:20:48.585664 17810 webserver.cc:460] Webserver started at http://127.29.229.62:39793/ using document root <none> and password file <none>
I20260811 22:20:48.586767 17810 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:48.586957 17810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:48.587442 17810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:20:48 dist-test-slave-jq4t krb5kdc[17081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486848, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
I20260811 22:20:48.664539 17810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "d673c6d2bbeb494e9965691e1d0f8ba9"
format_stamp: "Formatted at 2026-08-11 22:20:48 on dist-test-slave-jq4t"
server_key: "67c45211753f5920a72985fcb594bef6"
server_key_iv: "1195d534d4a09f7eec465caa4c3bb3ed"
server_key_version: "kuduclusterkey@0"
I20260811 22:20:48.666190 17810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "d673c6d2bbeb494e9965691e1d0f8ba9"
format_stamp: "Formatted at 2026-08-11 22:20:48 on dist-test-slave-jq4t"
server_key: "67c45211753f5920a72985fcb594bef6"
server_key_iv: "1195d534d4a09f7eec465caa4c3bb3ed"
server_key_version: "kuduclusterkey@0"
I20260811 22:20:48.675046 17810 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260811 22:20:48.697736 17832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:48.699739 17810 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 22:20:48.700094 17810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "d673c6d2bbeb494e9965691e1d0f8ba9"
format_stamp: "Formatted at 2026-08-11 22:20:48 on dist-test-slave-jq4t"
server_key: "67c45211753f5920a72985fcb594bef6"
server_key_iv: "1195d534d4a09f7eec465caa4c3bb3ed"
server_key_version: "kuduclusterkey@0"
I20260811 22:20:48.700377 17810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:48.735498 17810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:48.742437 17810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:48.742919 17810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:48.776674 17810 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:43923
I20260811 22:20:48.776691 17916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:43923 every 8 connection(s)
I20260811 22:20:48.778554 17810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:20:48.783074 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 17810
I20260811 22:20:48.783330 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:20:48.784206 30612 external_mini_cluster.cc:1428] Setting key 4dee783b5f15730a8d03afd69fbe94dc
I20260811 22:20:48.784989 17918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:48.817505 17918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9: Bootstrap starting.
I20260811 22:20:48.823266 17918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:48.824406 17918 log.cc:824] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:20:48 dist-test-slave-jq4t krb5kdc[17081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:20:48.828536 17918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9: No bootstrap required, opened a new log
I20260811 22:20:48.837935 17921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:48.787074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46948 (local address 127.29.229.62:43923)
0811 22:20:48.808564 (+ 21490us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:48.808585 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:48.809916 (+ 1331us) server_negotiation.cc:407] Connection header received
0811 22:20:48.812949 (+ 3033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:48.812994 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:48.813708 (+ 714us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:48.814267 (+ 559us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:48.816071 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:48.818353 (+ 2282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:48.820340 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:48.821439 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:48.821635 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:48.827207 (+ 5572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:48.827220 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:48.827240 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:48.827295 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:48.831840 (+ 4545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:48.833086 (+ 1246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:48.833098 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:48.833108 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:48.833347 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:48.833869 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:48.833875 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:48.833879 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:48.834778 (+ 899us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:48.835216 (+ 438us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:48.835715 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 22:20:48.836233 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21139,"thread_start_us":20834,"threads_started":1}
I20260811 22:20:48.840695 17918 raft_consensus.cc:348] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d673c6d2bbeb494e9965691e1d0f8ba9" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43923 } }
I20260811 22:20:48.841370 17918 raft_consensus.cc:374] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:48.841521 17918 raft_consensus.cc:729] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d673c6d2bbeb494e9965691e1d0f8ba9, State: Initialized, Role: FOLLOWER
I20260811 22:20:48.842413 17918 consensus_queue.cc:260] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d673c6d2bbeb494e9965691e1d0f8ba9" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43923 } }
I20260811 22:20:48.842867 17918 raft_consensus.cc:388] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:20:48.843062 17918 raft_consensus.cc:482] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:20:48.843343 17918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:48.847600 17918 raft_consensus.cc:504] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d673c6d2bbeb494e9965691e1d0f8ba9" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43923 } }
I20260811 22:20:48.848444 17918 leader_election.cc:302] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d673c6d2bbeb494e9965691e1d0f8ba9; no voters:
I20260811 22:20:48.849349 17918 leader_election.cc:288] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:20:48.849507 17924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:48.850467 17924 raft_consensus.cc:686] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9 [term 1 LEADER]: Becoming Leader. State: Replica: d673c6d2bbeb494e9965691e1d0f8ba9, State: Running, Role: LEADER
I20260811 22:20:48.851255 17924 consensus_queue.cc:237] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d673c6d2bbeb494e9965691e1d0f8ba9" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43923 } }
I20260811 22:20:48.852064 17918 sys_catalog.cc:564] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:20:48.854314 17928 sys_catalog.cc:455] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d673c6d2bbeb494e9965691e1d0f8ba9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d673c6d2bbeb494e9965691e1d0f8ba9" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43923 } } }
I20260811 22:20:48.854739 17928 sys_catalog.cc:458] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:48.854964 17929 sys_catalog.cc:455] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d673c6d2bbeb494e9965691e1d0f8ba9. Latest consensus state: current_term: 1 leader_uuid: "d673c6d2bbeb494e9965691e1d0f8ba9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d673c6d2bbeb494e9965691e1d0f8ba9" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43923 } } }
I20260811 22:20:48.855312 17929 sys_catalog.cc:458] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:20:48.858330 17932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:20:48.865830 17932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:20:48.873255 17918 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 22:20:48.887944 17932 catalog_manager.cc:1347] Generated new cluster ID: fc45cd6b1cf148e6b1586210bf40d49b
I20260811 22:20:48.888209 17932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:20:48.911880 17932 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:20:48.913481 17932 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:20:48.922923 17932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9: Generated new TSK 0
I20260811 22:20:48.923908 17932 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
I20260811 22:20:49.797667 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.229.20:59049/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43923
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:43963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:20:50 dist-test-slave-jq4t krb5kdc[17081](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486850, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 22:20:50.300374 17997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:50.300794 17997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:50.300983 17997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:50.310362 17997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:50.310531 17997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:50.310734 17997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:20:50.324355 17997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:43963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.229.20:59049/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:20:50.325897 17997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:50.327268 17997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:50.344609 18014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:50.345335 18013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:50.345438 18021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:50.352219 17997 server_base.cc:1034] running on GCE node
I20260811 22:20:50.353433 17997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:50.357718 17997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:50.359107 17997 hybrid_clock.cc:648] HybridClock initialized: now 1786486850358987 us; error 115 us; skew 500 ppm
I20260811 22:20:50.359612 17997 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:20:50 dist-test-slave-jq4t krb5kdc[17081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486850, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:50.364126 17997 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:20:50.376678 17997 webserver.cc:460] Webserver started at http://127.29.229.1:37051/ using document root <none> and password file <none>
I20260811 22:20:50.377784 17997 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:50.378033 17997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:50.378527 17997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:20:50 dist-test-slave-jq4t krb5kdc[17081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486850, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
I20260811 22:20:50.412251 17997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "a3d298d7f08f490d8a129ad83a83d2a1"
format_stamp: "Formatted at 2026-08-11 22:20:50 on dist-test-slave-jq4t"
server_key: "3095d471f1772abf49b3fd8da85a2f7c"
server_key_iv: "ed71a51104a875bdb2f71aaa9974a785"
server_key_version: "kuduclusterkey@0"
I20260811 22:20:50.414063 17997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "a3d298d7f08f490d8a129ad83a83d2a1"
format_stamp: "Formatted at 2026-08-11 22:20:50 on dist-test-slave-jq4t"
server_key: "3095d471f1772abf49b3fd8da85a2f7c"
server_key_iv: "ed71a51104a875bdb2f71aaa9974a785"
server_key_version: "kuduclusterkey@0"
I20260811 22:20:50.437568 17997 fs_manager.cc:691] Time spent creating directory manager: real 0.023s user 0.008s sys 0.012s
I20260811 22:20:50.471663 18031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:50.473975 17997 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.006s sys 0.005s
I20260811 22:20:50.474225 17997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "a3d298d7f08f490d8a129ad83a83d2a1"
format_stamp: "Formatted at 2026-08-11 22:20:50 on dist-test-slave-jq4t"
server_key: "3095d471f1772abf49b3fd8da85a2f7c"
server_key_iv: "ed71a51104a875bdb2f71aaa9974a785"
server_key_version: "kuduclusterkey@0"
I20260811 22:20:50.474536 17997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:50.499155 17997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:50.506346 17997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:50.506826 17997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:50.533529 17997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:50.548208 17997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:50.548416 17997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 22:20:50.548646 17997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:50.548751 17997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:20:50 dist-test-slave-jq4t krb5kdc[17081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486850, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
I20260811 22:20:50.675186 17997 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:39483
I20260811 22:20:50.675212 18211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:39483 every 8 connection(s)
I20260811 22:20:50.676844 17997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:20:50.679036 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 17997
I20260811 22:20:50.679309 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:20:50.680349 30612 external_mini_cluster.cc:1428] Setting key 1abffe5bdb5d00956399d7a782700556
Aug 11 22:20:50 dist-test-slave-jq4t krb5kdc[17081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486850, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.229.2@KRBTEST.COM; defaulting to no policy
I20260811 22:20:50.741447 18221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:50.707873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:43627 (local address 127.29.229.62:43923)
0811 22:20:50.715065 (+ 7192us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:50.715072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:50.715101 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:20:50.716801 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:50.716807 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:50.716912 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:50.717053 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:50.719354 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:50.720681 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:50.723689 (+ 3008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:50.724446 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:50.724628 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:50.730701 (+ 6073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:50.730711 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:50.730719 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:50.730774 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:50.735398 (+ 4624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:50.736792 (+ 1394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:50.736812 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:50.736819 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:50.737029 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:50.737716 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:50.737724 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:50.737730 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:50.738186 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:50.738339 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:50.740689 (+ 2350us) server_negotiation.cc:299] Negotiation successful
0811 22:20:50.740972 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6923,"thread_start_us":8654,"threads_started":1}
I20260811 22:20:50.742595 18222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:50.708885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43923 (local address 127.29.229.1:43627)
0811 22:20:50.714479 (+ 5594us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:50.714544 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:50.716378 (+ 1834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:50.717329 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:50.717343 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:50.718226 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:50.719113 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:50.719143 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:50.720886 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:50.720894 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:50.723492 (+ 2598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:50.723512 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:50.724772 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:50.724786 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:50.725287 (+ 501us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:50.726253 (+ 966us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:50.726293 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:50.730156 (+ 3863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:50.735676 (+ 5520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:50.735690 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:50.735716 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:50.736594 (+ 878us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:50.737202 (+ 608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:50.737211 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:50.737217 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:50.737539 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:50.738449 (+ 910us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:50.738460 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:50.739016 (+ 556us) client_negotiation.cc:769] Sending connection context
0811 22:20:50.739382 (+ 366us) client_negotiation.cc:240] Negotiation successful
0811 22:20:50.740060 (+ 678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5255,"thread_start_us":4964,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.2@KRBTEST.COM" created.
I20260811 22:20:50.744503 18212 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43923
I20260811 22:20:50.744870 18212 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:50.745666 18212 heartbeater.cc:511] Master 127.29.229.62:43923 requested a full tablet report, sending...
I20260811 22:20:50.752101 17855 ts_manager.cc:194] Registered new tserver with Master: a3d298d7f08f490d8a129ad83a83d2a1 (127.29.229.1:39483)
I20260811 22:20:50.754555 17855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:43627
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
W20260811 22:20:50.799713 30612 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.29.229.2: real 0.113s user 0.000s sys 0.037s
I20260811 22:20:50.800714 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.229.20:59049/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43923
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:43963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:20:50 dist-test-slave-jq4t krb5kdc[17081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486850, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
W20260811 22:20:51.356902 18226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:51.357348 18226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:51.357513 18226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:51.369655 18226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:51.369810 18226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:51.369988 18226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:20:51.386593 18226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:43963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.229.20:59049/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:20:51.388815 18226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:51.391192 18226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:51.409596 18250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:51.409567 18252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:51.410454 18254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:51.410961 18226 server_base.cc:1034] running on GCE node
I20260811 22:20:51.411729 18226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:51.413556 18226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:51.414943 18226 hybrid_clock.cc:648] HybridClock initialized: now 1786486851414863 us; error 52 us; skew 500 ppm
I20260811 22:20:51.415571 18226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:20:51 dist-test-slave-jq4t krb5kdc[17081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486851, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:51.422865 18226 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:20:51.424726 18226 webserver.cc:460] Webserver started at http://127.29.229.2:36757/ using document root <none> and password file <none>
I20260811 22:20:51.425508 18226 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:51.425670 18226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:51.426018 18226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:20:51 dist-test-slave-jq4t krb5kdc[17081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486851, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
I20260811 22:20:51.446559 18226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "7ea00267241a4388b4470e1128792a8e"
format_stamp: "Formatted at 2026-08-11 22:20:51 on dist-test-slave-jq4t"
server_key: "2e398891280e1cba707e2a7c84e51bd1"
server_key_iv: "928f123338110ea6615a3ca4a505e285"
server_key_version: "kuduclusterkey@0"
I20260811 22:20:51.447736 18226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "7ea00267241a4388b4470e1128792a8e"
format_stamp: "Formatted at 2026-08-11 22:20:51 on dist-test-slave-jq4t"
server_key: "2e398891280e1cba707e2a7c84e51bd1"
server_key_iv: "928f123338110ea6615a3ca4a505e285"
server_key_version: "kuduclusterkey@0"
I20260811 22:20:51.455232 18226 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.004s
I20260811 22:20:51.474610 18266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:51.476843 18226 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:20:51.477090 18226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "7ea00267241a4388b4470e1128792a8e"
format_stamp: "Formatted at 2026-08-11 22:20:51 on dist-test-slave-jq4t"
server_key: "2e398891280e1cba707e2a7c84e51bd1"
server_key_iv: "928f123338110ea6615a3ca4a505e285"
server_key_version: "kuduclusterkey@0"
I20260811 22:20:51.477361 18226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:51.496162 18226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:51.501713 18226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:51.502079 18226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:51.503364 18226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:51.504957 18226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:51.505064 18226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:51.505208 18226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:51.505292 18226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:51.550079 18226 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:35351
I20260811 22:20:51.550100 18444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:35351 every 8 connection(s)
I20260811 22:20:51.552124 18226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:20:51.557372 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 18226
I20260811 22:20:51.557793 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:20:51.558705 30612 external_mini_cluster.cc:1428] Setting key 0413a2bb022436905a540056aecf31fb
Aug 11 22:20:51 dist-test-slave-jq4t krb5kdc[17081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486851, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:20:51.586031 18449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:51.556892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:47225 (local address 127.29.229.62:43923)
0811 22:20:51.564999 (+ 8107us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:51.565007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:51.565035 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:20:51.565571 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:51.565577 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:51.565690 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:51.565832 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:51.567994 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:51.569167 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:51.571832 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:51.572506 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:51.572637 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:51.577580 (+ 4943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:51.577590 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:51.577598 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:51.577657 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:51.582018 (+ 4361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:51.582939 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:51.582947 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:51.582951 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:51.583094 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:51.583742 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:51.583748 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:51.583752 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:51.584153 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:51.584304 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:51.585372 (+ 1068us) server_negotiation.cc:299] Negotiation successful
0811 22:20:51.585624 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7957,"thread_start_us":7809,"threads_started":1}
I20260811 22:20:51.587455 18450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:51.557012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43923 (local address 127.29.229.2:47225)
0811 22:20:51.560939 (+ 3927us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:51.561094 (+ 155us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:51.565206 (+ 4112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:51.566170 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:51.566179 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:51.566833 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:51.567771 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:51.567800 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:51.569355 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:51.569363 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:51.571640 (+ 2277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:51.571659 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:51.572767 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:51.572777 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:51.573129 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:51.574057 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:51.574095 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:51.577253 (+ 3158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:51.582187 (+ 4934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:51.582197 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:51.582215 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:51.582798 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:51.583230 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:51.583236 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:51.583239 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:51.583592 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:51.584326 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:51.584337 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:51.584887 (+ 550us) client_negotiation.cc:769] Sending connection context
0811 22:20:51.585226 (+ 339us) client_negotiation.cc:240] Negotiation successful
0811 22:20:51.585857 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3376,"thread_start_us":3070,"threads_started":1}
I20260811 22:20:51.589257 18446 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43923
I20260811 22:20:51.589617 18446 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:51.590583 18446 heartbeater.cc:511] Master 127.29.229.62:43923 requested a full tablet report, sending...
I20260811 22:20:51.592731 17855 ts_manager.cc:194] Registered new tserver with Master: 7ea00267241a4388b4470e1128792a8e (127.29.229.2:35351)
I20260811 22:20:51.594010 17855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:47225
WARNING: no policy specified for kudu/127.29.229.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
I20260811 22:20:51.644181 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.229.20:59049/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43923
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:43963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:20:51.759963 18212 heartbeater.cc:503] Master 127.29.229.62:43923 was elected leader, sending a full tablet report...
W20260811 22:20:52.020709 18455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:20:52.021011 18455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:20:52.021175 18455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:20:52.028628 18455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:20:52.028738 18455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:20:52.028945 18455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:20:52.039280 18455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:43963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.229.20:59049/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:20:52.040848 18455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:20:52.042239 18455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:20:52.058174 18471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:52.058297 18464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:20:52.058274 18467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:20:52.059226 18455 server_base.cc:1034] running on GCE node
I20260811 22:20:52.060011 18455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:20:52.061225 18455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:20:52.062510 18455 hybrid_clock.cc:648] HybridClock initialized: now 1786486852062452 us; error 50 us; skew 500 ppm
I20260811 22:20:52.062897 18455 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:20:52 dist-test-slave-jq4t krb5kdc[17081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486852, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:20:52.066725 18455 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:20:52.068403 18455 webserver.cc:460] Webserver started at http://127.29.229.3:38715/ using document root <none> and password file <none>
I20260811 22:20:52.069181 18455 fs_manager.cc:359] Metadata directory not provided
I20260811 22:20:52.069352 18455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:20:52.069697 18455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:20:52 dist-test-slave-jq4t krb5kdc[17081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486852, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
I20260811 22:20:52.087486 18455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "88be7617368848dfa82663acedbbaa1e"
format_stamp: "Formatted at 2026-08-11 22:20:52 on dist-test-slave-jq4t"
server_key: "6886d244f879d56cf67c15b4e2905cac"
server_key_iv: "ceddd72f7019a3742db86524fb9fca45"
server_key_version: "kuduclusterkey@0"
I20260811 22:20:52.088551 18455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "88be7617368848dfa82663acedbbaa1e"
format_stamp: "Formatted at 2026-08-11 22:20:52 on dist-test-slave-jq4t"
server_key: "6886d244f879d56cf67c15b4e2905cac"
server_key_iv: "ceddd72f7019a3742db86524fb9fca45"
server_key_version: "kuduclusterkey@0"
I20260811 22:20:52.095003 18455 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 22:20:52.114671 18482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:52.116670 18455 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:20:52.116880 18455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "88be7617368848dfa82663acedbbaa1e"
format_stamp: "Formatted at 2026-08-11 22:20:52 on dist-test-slave-jq4t"
server_key: "6886d244f879d56cf67c15b4e2905cac"
server_key_iv: "ceddd72f7019a3742db86524fb9fca45"
server_key_version: "kuduclusterkey@0"
I20260811 22:20:52.117146 18455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:20:52.132787 18455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:20:52.138682 18455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:20:52.139119 18455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:20:52.140537 18455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:20:52.142567 18455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:20:52.142737 18455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:52.142938 18455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:20:52.143028 18455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:20:52.181901 18455 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:45169
I20260811 22:20:52.181989 18663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:45169 every 8 connection(s)
I20260811 22:20:52.183317 18455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:20:52.192898 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 18455
I20260811 22:20:52.193120 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:20:52.193578 30612 external_mini_cluster.cc:1428] Setting key 42acf86ed253ff46dc563f9ec8ba7686
Aug 11 22:20:52 dist-test-slave-jq4t krb5kdc[17081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486852, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:20:52.206079 18669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.186116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:39551 (local address 127.29.229.62:43923)
0811 22:20:52.186645 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:52.186653 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:52.187710 (+ 1057us) server_negotiation.cc:407] Connection header received
0811 22:20:52.189661 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:52.189670 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:52.189765 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:52.189921 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:52.191762 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.192572 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.194438 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.195119 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.195222 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.199317 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:52.199323 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:52.199327 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:52.199361 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:52.202599 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:52.203466 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:52.203473 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:52.203476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:52.203625 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:52.204113 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:52.204118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:52.204121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:52.204438 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:52.204679 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:52.205419 (+ 740us) server_negotiation.cc:299] Negotiation successful
0811 22:20:52.205696 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":226,"threads_started":1}
I20260811 22:20:52.207166 18672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.186800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43923 (local address 127.29.229.3:39551)
0811 22:20:52.187537 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:52.187575 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:52.189324 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:52.190226 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:52.190238 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:52.190873 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:20:52.191587 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:52.191604 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.192721 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.192727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:52.194241 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:52.194276 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.195359 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.195369 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:52.195655 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.196453 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:20:52.196485 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:20:52.199057 (+ 2572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:20:52.202771 (+ 3714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:52.202778 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:52.202806 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:52.203296 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:52.203753 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:20:52.203758 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:20:52.203762 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:20:52.203988 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:20:52.204657 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:20:52.204666 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:20:52.205028 (+ 362us) client_negotiation.cc:769] Sending connection context
0811 22:20:52.205329 (+ 301us) client_negotiation.cc:240] Negotiation successful
0811 22:20:52.205931 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":306,"threads_started":1}
I20260811 22:20:52.208792 18665 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43923
I20260811 22:20:52.209156 18665 heartbeater.cc:464] Registering TS with master...
I20260811 22:20:52.209946 18665 heartbeater.cc:511] Master 127.29.229.62:43923 requested a full tablet report, sending...
I20260811 22:20:52.211462 17854 ts_manager.cc:194] Registered new tserver with Master: 88be7617368848dfa82663acedbbaa1e (127.29.229.3:45169)
I20260811 22:20:52.212360 17854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:39551
I20260811 22:20:52.222448 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:20:52.240751 18669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.227696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46962 (local address 127.29.229.62:43923)
0811 22:20:52.227945 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:52.227955 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:52.228364 (+ 409us) server_negotiation.cc:407] Connection header received
0811 22:20:52.228613 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:52.228617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:52.228707 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:52.228799 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:52.230333 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.231100 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.232380 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.232862 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.232968 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.235395 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:52.235400 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:52.235403 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:52.235434 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:52.238143 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:52.238843 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:52.238847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:52.238849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:52.238983 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:52.239366 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:52.239369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:52.239371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:52.239663 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:52.239841 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:52.240174 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 22:20:52.240347 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:20:52.245477 17854 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:20:52.252928 17854 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:20:52.279266 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.266866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54668 (local address 127.29.229.3:45169)
0811 22:20:52.270586 (+ 3720us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:52.270594 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:52.270623 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:20:52.270701 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:52.270707 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:52.270966 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:52.271184 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:52.272663 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.274304 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.276773 (+ 2469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.277849 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.278019 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.278272 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:52.278738 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 22:20:52.278897 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3561,"thread_start_us":3414,"threads_started":1}
I20260811 22:20:52.279351 18685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.266113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:35351 (local address 127.0.0.1:52936)
0811 22:20:52.267100 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:52.267143 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:52.267334 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:52.271210 (+ 3876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:52.271218 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:52.271249 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:52.271872 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:52.271886 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.273706 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.273714 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:52.276354 (+ 2640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:52.276370 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.278396 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.278403 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:52.278585 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.278621 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:20:52.278694 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 22:20:52.278840 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"mutex_wait_us":82,"thread_start_us":1042,"threads_started":1}
I20260811 22:20:52.279537 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.269164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34826 (local address 127.29.229.1:39483)
0811 22:20:52.270977 (+ 1813us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:52.270983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:52.271005 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:20:52.271088 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:52.271092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:52.271331 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:52.271552 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:52.272439 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.274143 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.276851 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.278259 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.278406 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.278617 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:52.278980 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 22:20:52.279153 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1671,"thread_start_us":1493,"threads_started":1}
I20260811 22:20:52.279884 18697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.269246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:39483 (local address 127.0.0.1:34826)
0811 22:20:52.269857 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:52.269875 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:52.270080 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:52.271532 (+ 1452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:52.271536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:52.271556 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:52.272223 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:52.272236 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.274321 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.274326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:52.276673 (+ 2347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:52.276688 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.278518 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.278524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:52.278680 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.278707 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:20:52.278792 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 22:20:52.278930 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":2,"thread_start_us":346,"threads_started":1}
I20260811 22:20:52.280081 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.266385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:45169 (local address 127.0.0.1:54668)
0811 22:20:52.268058 (+ 1673us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:52.268084 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:52.268239 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:52.271497 (+ 3258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:52.271503 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:52.271531 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:52.272404 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:52.272418 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.274446 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.274452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:52.276573 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:52.276587 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.278055 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.278061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:52.278260 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.278339 (+ 79us) client_negotiation.cc:769] Sending connection context
0811 22:20:52.278498 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 22:20:52.278691 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1495,"mutex_wait_us":164,"thread_start_us":1320,"threads_started":1}
I20260811 22:20:52.280584 18686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.266362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52936 (local address 127.29.229.2:35351)
0811 22:20:52.270524 (+ 4162us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:52.270534 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:52.270565 (+ 31us) server_negotiation.cc:407] Connection header received
0811 22:20:52.270672 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:52.270680 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:52.270967 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:52.271184 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:52.272095 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.273572 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.276599 (+ 3027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.278106 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.279734 (+ 1628us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.279974 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:52.280049 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 22:20:52.280233 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3957,"thread_start_us":2164,"threads_started":1}
I20260811 22:20:52.283989 18562 tablet_service.cc:1467] Processing CreateTablet for tablet bf75e0ad68b1439c8db5614971188ac8 (DEFAULT_TABLE table=test-table [id=0c486e664fb848c497d23c1cba6e3071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:52.284112 18105 tablet_service.cc:1467] Processing CreateTablet for tablet bf75e0ad68b1439c8db5614971188ac8 (DEFAULT_TABLE table=test-table [id=0c486e664fb848c497d23c1cba6e3071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:52.284113 18348 tablet_service.cc:1467] Processing CreateTablet for tablet bf75e0ad68b1439c8db5614971188ac8 (DEFAULT_TABLE table=test-table [id=0c486e664fb848c497d23c1cba6e3071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:20:52.285477 18562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf75e0ad68b1439c8db5614971188ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:52.285545 18348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf75e0ad68b1439c8db5614971188ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:52.285629 18105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf75e0ad68b1439c8db5614971188ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:20:52.297575 18701 tablet_bootstrap.cc:492] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e: Bootstrap starting.
I20260811 22:20:52.301276 18702 tablet_bootstrap.cc:492] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1: Bootstrap starting.
I20260811 22:20:52.302075 18701 tablet_bootstrap.cc:654] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:52.303095 18703 tablet_bootstrap.cc:492] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e: Bootstrap starting.
I20260811 22:20:52.303557 18701 log.cc:824] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:52.306305 18702 tablet_bootstrap.cc:654] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:52.306599 18703 tablet_bootstrap.cc:654] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e: Neither blocks nor log segments found. Creating new log.
I20260811 22:20:52.307204 18701 tablet_bootstrap.cc:492] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e: No bootstrap required, opened a new log
I20260811 22:20:52.307518 18701 ts_tablet_manager.cc:1397] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260811 22:20:52.307699 18702 log.cc:824] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:52.308455 18703 log.cc:824] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:20:52.315542 18703 tablet_bootstrap.cc:492] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e: No bootstrap required, opened a new log
I20260811 22:20:52.315726 18702 tablet_bootstrap.cc:492] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1: No bootstrap required, opened a new log
I20260811 22:20:52.315920 18703 ts_tablet_manager.cc:1397] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260811 22:20:52.316026 18702 ts_tablet_manager.cc:1397] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.000s
I20260811 22:20:52.319571 18701 raft_consensus.cc:348] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea00267241a4388b4470e1128792a8e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35351 } } peers { permanent_uuid: "88be7617368848dfa82663acedbbaa1e" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45169 } } peers { permanent_uuid: "a3d298d7f08f490d8a129ad83a83d2a1" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39483 } }
I20260811 22:20:52.320065 18701 raft_consensus.cc:374] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:52.320165 18701 raft_consensus.cc:729] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88be7617368848dfa82663acedbbaa1e, State: Initialized, Role: FOLLOWER
I20260811 22:20:52.321012 18701 consensus_queue.cc:260] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea00267241a4388b4470e1128792a8e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35351 } } peers { permanent_uuid: "88be7617368848dfa82663acedbbaa1e" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45169 } } peers { permanent_uuid: "a3d298d7f08f490d8a129ad83a83d2a1" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39483 } }
I20260811 22:20:52.322274 18665 heartbeater.cc:503] Master 127.29.229.62:43923 was elected leader, sending a full tablet report...
I20260811 22:20:52.323168 18701 ts_tablet_manager.cc:1428] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260811 22:20:52.325484 18703 raft_consensus.cc:348] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea00267241a4388b4470e1128792a8e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35351 } } peers { permanent_uuid: "88be7617368848dfa82663acedbbaa1e" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45169 } } peers { permanent_uuid: "a3d298d7f08f490d8a129ad83a83d2a1" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39483 } }
I20260811 22:20:52.325987 18703 raft_consensus.cc:374] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:52.326104 18703 raft_consensus.cc:729] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ea00267241a4388b4470e1128792a8e, State: Initialized, Role: FOLLOWER
I20260811 22:20:52.326784 18703 consensus_queue.cc:260] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea00267241a4388b4470e1128792a8e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35351 } } peers { permanent_uuid: "88be7617368848dfa82663acedbbaa1e" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45169 } } peers { permanent_uuid: "a3d298d7f08f490d8a129ad83a83d2a1" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39483 } }
I20260811 22:20:52.327149 18702 raft_consensus.cc:348] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea00267241a4388b4470e1128792a8e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35351 } } peers { permanent_uuid: "88be7617368848dfa82663acedbbaa1e" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45169 } } peers { permanent_uuid: "a3d298d7f08f490d8a129ad83a83d2a1" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39483 } }
I20260811 22:20:52.327620 18702 raft_consensus.cc:374] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:20:52.327739 18702 raft_consensus.cc:729] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d298d7f08f490d8a129ad83a83d2a1, State: Initialized, Role: FOLLOWER
I20260811 22:20:52.328352 18446 heartbeater.cc:503] Master 127.29.229.62:43923 was elected leader, sending a full tablet report...
I20260811 22:20:52.328348 18702 consensus_queue.cc:260] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea00267241a4388b4470e1128792a8e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35351 } } peers { permanent_uuid: "88be7617368848dfa82663acedbbaa1e" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45169 } } peers { permanent_uuid: "a3d298d7f08f490d8a129ad83a83d2a1" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39483 } }
I20260811 22:20:52.328846 18703 ts_tablet_manager.cc:1428] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260811 22:20:52.330626 18702 ts_tablet_manager.cc:1428] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1: Time spent starting tablet: real 0.014s user 0.008s sys 0.008s
W20260811 22:20:52.435542 18668 tablet.cc:2369] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:20:52.463593 18214 tablet.cc:2369] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:20:52.555626 18447 tablet.cc:2369] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:20:52.653963 18712 raft_consensus.cc:482] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:20:52.654338 18712 raft_consensus.cc:504] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea00267241a4388b4470e1128792a8e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35351 } } peers { permanent_uuid: "88be7617368848dfa82663acedbbaa1e" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45169 } } peers { permanent_uuid: "a3d298d7f08f490d8a129ad83a83d2a1" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39483 } }
I20260811 22:20:52.655391 18715 raft_consensus.cc:482] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:20:52.655709 18715 raft_consensus.cc:504] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea00267241a4388b4470e1128792a8e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35351 } } peers { permanent_uuid: "88be7617368848dfa82663acedbbaa1e" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45169 } } peers { permanent_uuid: "a3d298d7f08f490d8a129ad83a83d2a1" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39483 } }
I20260811 22:20:52.656518 18712 leader_election.cc:288] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88be7617368848dfa82663acedbbaa1e (127.29.229.3:45169), a3d298d7f08f490d8a129ad83a83d2a1 (127.29.229.1:39483)
I20260811 22:20:52.663780 18715 leader_election.cc:288] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ea00267241a4388b4470e1128792a8e (127.29.229.2:35351), 88be7617368848dfa82663acedbbaa1e (127.29.229.3:45169)
I20260811 22:20:52.675706 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.657031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:44921 (local address 127.29.229.3:45169)
0811 22:20:52.657275 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:52.657285 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:52.657840 (+ 555us) server_negotiation.cc:407] Connection header received
0811 22:20:52.658190 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:52.658198 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:52.658287 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:52.658364 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:52.666533 (+ 8169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.668030 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.672177 (+ 4147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.674307 (+ 2130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.674577 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.674670 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:52.675245 (+ 575us) server_negotiation.cc:299] Negotiation successful
0811 22:20:52.675429 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":75}
I20260811 22:20:52.680068 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.670189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:35557 (local address 127.29.229.1:39483)
0811 22:20:52.670538 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:52.670544 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:52.670568 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:20:52.670647 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:52.670665 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:52.670740 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:52.671033 (+ 293us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:52.673398 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.675602 (+ 2204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.677519 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.678764 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.678937 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.679015 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:52.679616 (+ 601us) server_negotiation.cc:299] Negotiation successful
0811 22:20:52.679776 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":88,"server-negotiator.queue_time_us":51}
I20260811 22:20:52.680814 18686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.669303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:45217 (local address 127.29.229.2:35351)
0811 22:20:52.669490 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:52.669497 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:52.669513 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:20:52.669569 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:52.669575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:52.669671 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:52.669767 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:52.671008 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.672405 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.678214 (+ 5809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.679463 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.679632 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.679699 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:52.680301 (+ 602us) server_negotiation.cc:299] Negotiation successful
0811 22:20:52.680549 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:20:52.681562 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.657816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:39483 (local address 127.29.229.2:35557)
0811 22:20:52.669399 (+ 11583us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:52.669417 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:52.669553 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:52.671335 (+ 1782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:52.671340 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:52.671370 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:52.673081 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:52.673109 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.675772 (+ 2663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.675780 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:52.677334 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:52.677350 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.679185 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.679192 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:52.679412 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.679448 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:20:52.680725 (+ 1277us) client_negotiation.cc:240] Negotiation successful
0811 22:20:52.681123 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11430,"thread_start_us":11189,"threads_started":1}
I20260811 22:20:52.681598 18719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.657823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:35351 (local address 127.29.229.1:45217)
0811 22:20:52.667799 (+ 9976us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:52.667835 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:52.668020 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:52.669997 (+ 1977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:52.670002 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:52.670034 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:52.670812 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:52.670827 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.672656 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.672661 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:52.678007 (+ 5346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:52.678030 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.679870 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.679877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:52.680073 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.680123 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 22:20:52.681011 (+ 888us) client_negotiation.cc:240] Negotiation successful
0811 22:20:52.681160 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9775,"thread_start_us":10313,"threads_started":1}
I20260811 22:20:52.682818 18377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf75e0ad68b1439c8db5614971188ac8" candidate_uuid: "a3d298d7f08f490d8a129ad83a83d2a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ea00267241a4388b4470e1128792a8e" is_pre_election: true
I20260811 22:20:52.683314 18377 raft_consensus.cc:2454] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d298d7f08f490d8a129ad83a83d2a1 in term 0.
I20260811 22:20:52.683163 18140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf75e0ad68b1439c8db5614971188ac8" candidate_uuid: "7ea00267241a4388b4470e1128792a8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d298d7f08f490d8a129ad83a83d2a1" is_pre_election: true
I20260811 22:20:52.683656 18140 raft_consensus.cc:2454] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ea00267241a4388b4470e1128792a8e in term 0.
I20260811 22:20:52.684048 18033 leader_election.cc:302] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ea00267241a4388b4470e1128792a8e, a3d298d7f08f490d8a129ad83a83d2a1; no voters:
I20260811 22:20:52.684940 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.658653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:53007 (local address 127.29.229.3:45169)
0811 22:20:52.668867 (+ 10214us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:52.668877 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:52.670321 (+ 1444us) server_negotiation.cc:407] Connection header received
0811 22:20:52.670829 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:52.670834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:52.670924 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:52.671003 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:20:52.671914 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.674892 (+ 2978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.680458 (+ 5566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.682003 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.682153 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.682220 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:52.684377 (+ 2157us) server_negotiation.cc:299] Negotiation successful
0811 22:20:52.684569 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10042,"thread_start_us":10223,"threads_started":1}
I20260811 22:20:52.685002 18720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.658125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:45169 (local address 127.29.229.1:53007)
0811 22:20:52.670209 (+ 12084us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:52.670235 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:52.670590 (+ 355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:52.671306 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:52.671311 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:52.671351 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:52.671730 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:52.671742 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.675233 (+ 3491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.675239 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:52.680243 (+ 5004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:52.680259 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.682563 (+ 2304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.682570 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:52.684071 (+ 1501us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.684111 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 22:20:52.684253 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 22:20:52.684413 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11907,"mutex_wait_us":1,"thread_start_us":11702,"threads_started":1}
I20260811 22:20:52.685164 18716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.656938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:45169 (local address 127.29.229.2:44921)
0811 22:20:52.657710 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:52.657739 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 22:20:52.657993 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:20:52.664873 (+ 6880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:20:52.664881 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:20:52.664930 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:20:52.665458 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:52.666217 (+ 759us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.668319 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.668328 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:52.671844 (+ 3516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:20:52.671862 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.674829 (+ 2967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.674836 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:20:52.675027 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.675064 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 22:20:52.684602 (+ 9538us) client_negotiation.cc:240] Negotiation successful
0811 22:20:52.684781 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":424,"threads_started":1}
I20260811 22:20:52.685073 18275 leader_election.cc:302] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ea00267241a4388b4470e1128792a8e, a3d298d7f08f490d8a129ad83a83d2a1; no voters:
I20260811 22:20:52.684567 18715 raft_consensus.cc:2781] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:20:52.685470 18715 raft_consensus.cc:482] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:20:52.685561 18715 raft_consensus.cc:3037] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:52.685653 18712 raft_consensus.cc:2781] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:20:52.686126 18712 raft_consensus.cc:482] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:20:52.686561 18712 raft_consensus.cc:3037] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:52.688081 18592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf75e0ad68b1439c8db5614971188ac8" candidate_uuid: "a3d298d7f08f490d8a129ad83a83d2a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88be7617368848dfa82663acedbbaa1e" is_pre_election: true
I20260811 22:20:52.688983 18715 raft_consensus.cc:504] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea00267241a4388b4470e1128792a8e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35351 } } peers { permanent_uuid: "88be7617368848dfa82663acedbbaa1e" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45169 } } peers { permanent_uuid: "a3d298d7f08f490d8a129ad83a83d2a1" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39483 } }
I20260811 22:20:52.689087 18592 raft_consensus.cc:2454] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d298d7f08f490d8a129ad83a83d2a1 in term 0.
I20260811 22:20:52.689728 18715 leader_election.cc:288] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [CANDIDATE]: Term 1 election: Requested vote from peers 7ea00267241a4388b4470e1128792a8e (127.29.229.2:35351), 88be7617368848dfa82663acedbbaa1e (127.29.229.3:45169)
I20260811 22:20:52.690332 18377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf75e0ad68b1439c8db5614971188ac8" candidate_uuid: "a3d298d7f08f490d8a129ad83a83d2a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ea00267241a4388b4470e1128792a8e"
I20260811 22:20:52.690529 18592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf75e0ad68b1439c8db5614971188ac8" candidate_uuid: "a3d298d7f08f490d8a129ad83a83d2a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88be7617368848dfa82663acedbbaa1e"
I20260811 22:20:52.690747 18592 raft_consensus.cc:3037] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:20:52.691319 18590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf75e0ad68b1439c8db5614971188ac8" candidate_uuid: "7ea00267241a4388b4470e1128792a8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88be7617368848dfa82663acedbbaa1e" is_pre_election: true
I20260811 22:20:52.690987 18712 raft_consensus.cc:504] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea00267241a4388b4470e1128792a8e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35351 } } peers { permanent_uuid: "88be7617368848dfa82663acedbbaa1e" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45169 } } peers { permanent_uuid: "a3d298d7f08f490d8a129ad83a83d2a1" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39483 } }
I20260811 22:20:52.692243 18377 raft_consensus.cc:2379] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a3d298d7f08f490d8a129ad83a83d2a1 in current term 1: Already voted for candidate 7ea00267241a4388b4470e1128792a8e in this term.
I20260811 22:20:52.692541 18712 leader_election.cc:288] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [CANDIDATE]: Term 1 election: Requested vote from peers 88be7617368848dfa82663acedbbaa1e (127.29.229.3:45169), a3d298d7f08f490d8a129ad83a83d2a1 (127.29.229.1:39483)
I20260811 22:20:52.693280 18590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf75e0ad68b1439c8db5614971188ac8" candidate_uuid: "7ea00267241a4388b4470e1128792a8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88be7617368848dfa82663acedbbaa1e"
I20260811 22:20:52.693722 18140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf75e0ad68b1439c8db5614971188ac8" candidate_uuid: "7ea00267241a4388b4470e1128792a8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d298d7f08f490d8a129ad83a83d2a1"
I20260811 22:20:52.694016 18140 raft_consensus.cc:2379] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7ea00267241a4388b4470e1128792a8e in current term 1: Already voted for candidate a3d298d7f08f490d8a129ad83a83d2a1 in this term.
I20260811 22:20:52.694113 18592 raft_consensus.cc:2454] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d298d7f08f490d8a129ad83a83d2a1 in term 1.
I20260811 22:20:52.694526 18275 leader_election.cc:302] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ea00267241a4388b4470e1128792a8e; no voters: 88be7617368848dfa82663acedbbaa1e, a3d298d7f08f490d8a129ad83a83d2a1
I20260811 22:20:52.694550 18036 leader_election.cc:302] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88be7617368848dfa82663acedbbaa1e, a3d298d7f08f490d8a129ad83a83d2a1; no voters: 7ea00267241a4388b4470e1128792a8e
I20260811 22:20:52.694805 18712 raft_consensus.cc:2726] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260811 22:20:52.695073 18715 raft_consensus.cc:2781] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:20:52.695557 18715 raft_consensus.cc:686] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [term 1 LEADER]: Becoming Leader. State: Replica: a3d298d7f08f490d8a129ad83a83d2a1, State: Running, Role: LEADER
I20260811 22:20:52.696077 18715 consensus_queue.cc:237] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea00267241a4388b4470e1128792a8e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35351 } } peers { permanent_uuid: "88be7617368848dfa82663acedbbaa1e" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45169 } } peers { permanent_uuid: "a3d298d7f08f490d8a129ad83a83d2a1" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39483 } }
I20260811 22:20:52.700783 17854 catalog_manager.cc:5495] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3d298d7f08f490d8a129ad83a83d2a1 (127.29.229.1). New cstate: current_term: 1 leader_uuid: "a3d298d7f08f490d8a129ad83a83d2a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea00267241a4388b4470e1128792a8e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88be7617368848dfa82663acedbbaa1e" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3d298d7f08f490d8a129ad83a83d2a1" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39483 } health_report { overall_health: HEALTHY } } }
I20260811 22:20:52.740228 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.734010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34830 (local address 127.29.229.1:39483)
0811 22:20:52.734247 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:52.734254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:52.734384 (+ 130us) server_negotiation.cc:407] Connection header received
0811 22:20:52.734667 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:52.734671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:52.734753 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:52.734924 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:20:52.735373 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.736129 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.737893 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.738339 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.738482 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.739028 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:20:52.739559 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:20:52.739722 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:52.739806 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 22:20:52.739896 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 22:20:52.749094 18592 raft_consensus.cc:1261] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e [term 1 FOLLOWER]: Refusing update from remote peer a3d298d7f08f490d8a129ad83a83d2a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:20:52.749128 18377 raft_consensus.cc:1261] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [term 1 FOLLOWER]: Refusing update from remote peer a3d298d7f08f490d8a129ad83a83d2a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:20:52.749984 18722 consensus_queue.cc:1035] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88be7617368848dfa82663acedbbaa1e" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 45169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:20:52.750345 18715 consensus_queue.cc:1035] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ea00267241a4388b4470e1128792a8e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:20:52.761451 18726 mvcc.cc:205] Tried to move back new op lower bound from 7317450148849930240 to 7317450148647366656. Current Snapshot: MvccSnapshot[applied={T|T < 7317450148849930240}]
I20260811 22:20:52.777580 17854 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:20:52.778051 17854 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:20:52.783039 17854 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d673c6d2bbeb494e9965691e1d0f8ba9: Sending DeleteTablet for 3 replicas of tablet bf75e0ad68b1439c8db5614971188ac8
I20260811 22:20:52.784358 18348 tablet_service.cc:1514] Processing DeleteTablet for tablet bf75e0ad68b1439c8db5614971188ac8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:52 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:52936
I20260811 22:20:52.784394 18562 tablet_service.cc:1514] Processing DeleteTablet for tablet bf75e0ad68b1439c8db5614971188ac8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:52 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:54668
I20260811 22:20:52.787386 18105 tablet_service.cc:1514] Processing DeleteTablet for tablet bf75e0ad68b1439c8db5614971188ac8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:20:52 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:34826
I20260811 22:20:52.796836 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 17997
I20260811 22:20:52.797079 18740 tablet_replica.cc:330] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e: stopping tablet replica
I20260811 22:20:52.797619 18740 raft_consensus.cc:2229] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:20:52.798209 18740 raft_consensus.cc:2258] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:20:52.803053 18740 ts_tablet_manager.cc:1905] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:20:52.814623 18743 tablet_replica.cc:330] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1: stopping tablet replica
I20260811 22:20:52.815150 18743 raft_consensus.cc:2229] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:20:52.815918 18743 raft_consensus.cc:2258] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:20:52.819288 18743 ts_tablet_manager.cc:1905] T bf75e0ad68b1439c8db5614971188ac8 P a3d298d7f08f490d8a129ad83a83d2a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:20:52.820750 18741 tablet_replica.cc:330] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e: stopping tablet replica
I20260811 22:20:52.821275 18741 raft_consensus.cc:2229] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:20:52.822072 18741 raft_consensus.cc:2258] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:20:52.825210 18741 ts_tablet_manager.cc:1905] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:20:52.825502 18740 ts_tablet_manager.cc:1918] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:20:52.825685 18740 log.cc:1196] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/bf75e0ad68b1439c8db5614971188ac8
I20260811 22:20:52.826252 18740 ts_tablet_manager.cc:1939] T bf75e0ad68b1439c8db5614971188ac8 P 7ea00267241a4388b4470e1128792a8e: Deleting consensus metadata
I20260811 22:20:52.828864 17836 catalog_manager.cc:4841] TS 7ea00267241a4388b4470e1128792a8e (127.29.229.2:35351): tablet bf75e0ad68b1439c8db5614971188ac8 (table test-table [id=0c486e664fb848c497d23c1cba6e3071]) successfully deleted
Aug 11 22:20:52 dist-test-slave-jq4t krb5kdc[17081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:20:52.833231 18741 ts_tablet_manager.cc:1918] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:20:52.833429 18741 log.cc:1196] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/bf75e0ad68b1439c8db5614971188ac8
I20260811 22:20:52.834033 18741 ts_tablet_manager.cc:1939] T bf75e0ad68b1439c8db5614971188ac8 P 88be7617368848dfa82663acedbbaa1e: Deleting consensus metadata
I20260811 22:20:52.836193 17837 catalog_manager.cc:4841] TS 88be7617368848dfa82663acedbbaa1e (127.29.229.3:45169): tablet bf75e0ad68b1439c8db5614971188ac8 (table test-table [id=0c486e664fb848c497d23c1cba6e3071]) successfully deleted
I20260811 22:20:52.837713 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.798131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34840 (local address 127.29.229.1:39483)
0811 22:20:52.798348 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:52.798356 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:52.817009 (+ 18653us) server_negotiation.cc:407] Connection header received
0811 22:20:52.817243 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:52.817249 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:52.817333 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:52.817440 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:52.819286 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.820420 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.822780 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.823568 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.823673 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.830583 (+ 6910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:52.830599 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:52.830614 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:52.830651 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:52.834458 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:52.835213 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:52.835221 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:52.835223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:52.835348 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:52.835757 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:52.835762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:52.835766 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:52.836192 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:52.836594 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:52.836976 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 22:20:52.837233 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:20:52.838464 18205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:20:52.944854 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 17997
W20260811 22:20:52.971446 17837 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.229.1:39483 (error 108)
I20260811 22:20:52.971668 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 18226
I20260811 22:20:52.974076 18748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.973107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:39483 (local address 127.0.0.1:34846)
0811 22:20:52.973789 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:52.973896 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":447,"threads_started":1}
W20260811 22:20:52.974363 17837 catalog_manager.cc:4584] TS a3d298d7f08f490d8a129ad83a83d2a1 (127.29.229.1:39483): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf75e0ad68b1439c8db5614971188ac8: Network error: Client connection negotiation failed: client connection to 127.29.229.1:39483: connect: Connection refused (error 111)
Aug 11 22:20:52 dist-test-slave-jq4t krb5kdc[17081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.2@KRBTEST.COM
I20260811 22:20:52.990893 18686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:52.972574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52946 (local address 127.29.229.2:35351)
0811 22:20:52.972760 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:52.972768 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:52.972843 (+ 75us) server_negotiation.cc:407] Connection header received
0811 22:20:52.973030 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:52.973035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:52.973119 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:52.973243 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:52.974738 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.975452 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.976945 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:52.977352 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:52.977449 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:52.982639 (+ 5190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:52.982663 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:52.982688 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:52.982757 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:52.986408 (+ 3651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:52.987908 (+ 1500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:52.987918 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:52.987921 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:52.988124 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:52.989073 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:52.989077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:52.989080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:52.989557 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:52.989811 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:52.990274 (+ 463us) server_negotiation.cc:299] Negotiation successful
0811 22:20:52.990516 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:20:52.991547 18439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:20:53.028407 18748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:53.027925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:39483 (local address 127.0.0.1:34860)
0811 22:20:53.028161 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:53.028259 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 22:20:53.030010 18748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:53.029464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:39483 (local address 127.0.0.1:34864)
0811 22:20:53.029771 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:53.029857 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260811 22:20:53.088646 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 18226
I20260811 22:20:53.093103 18748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:53.092644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:39483 (local address 127.0.0.1:34874)
0811 22:20:53.092852 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:53.092955 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 22:20:53.094151 18748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:53.093779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:39483 (local address 127.0.0.1:34890)
0811 22:20:53.093936 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:53.094016 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 22:20:53.121618 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 18455
Aug 11 22:20:53 dist-test-slave-jq4t krb5kdc[17081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.3@KRBTEST.COM
I20260811 22:20:53.140236 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:53.122448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54670 (local address 127.29.229.3:45169)
0811 22:20:53.122670 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 22:20:53.122676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:20:53.122786 (+ 110us) server_negotiation.cc:407] Connection header received
0811 22:20:53.123016 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:20:53.123020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:20:53.123090 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:20:53.123204 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:20:53.124752 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:53.125488 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:53.126910 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:20:53.127339 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:20:53.127484 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:20:53.133532 (+ 6048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:20:53.133553 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:20:53.133570 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:20:53.133622 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:20:53.136995 (+ 3373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:53.138007 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:53.138015 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:53.138017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:53.138198 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:20:53.138621 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:20:53.138624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:20:53.138626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:20:53.138986 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:20:53.139220 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 22:20:53.139649 (+ 429us) server_negotiation.cc:299] Negotiation successful
0811 22:20:53.139852 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:20:53.140949 18653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:20:53.175172 18748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:53.174514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:39483 (local address 127.0.0.1:34904)
0811 22:20:53.174822 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:53.174940 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260811 22:20:53.176971 18748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:53.176309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:39483 (local address 127.0.0.1:34910)
0811 22:20:53.176658 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:53.176761 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260811 22:20:53.236738 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 18455
I20260811 22:20:53.261202 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 17810
I20260811 22:20:53.262641 17907 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:20:53.383109 18748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:20:53.381796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:39483 (local address 127.0.0.1:34912)
0811 22:20:53.382566 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 22:20:53.382818 (+ 252us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392}
I20260811 22:20:53.383244 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 17810
I20260811 22:20:53.423951 30612 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 22:20:53.771003 30612 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 22:20:53.771389 30612 mini_ranger.cc:66] Stopping Ranger...
I20260811 22:20:54.128705 30612 mini_ranger.cc:68] Stopped Ranger
2026-08-11 22:20:54.129 UTC [17133] LOG: received smart shutdown request
2026-08-11 22:20:54.133 UTC [17133] LOG: background worker "logical replication launcher" (PID 17147) exited with exit code 1
2026-08-11 22:20:54.133 UTC [17140] LOG: shutting down
2026-08-11 22:20:54.284 UTC [17133] LOG: database system is shut down
2026-08-11T22:20:54Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (57544 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:20:54 dist-test-slave-jq4t krb5kdc[18766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:20:54 dist-test-slave-jq4t krb5kdc[18766](info): set up 2 sockets
Aug 11 22:20:54 dist-test-slave-jq4t krb5kdc[18766](info): commencing operation
krb5kdc: starting...
W20260811 22:20:55.872591 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.385s user 0.000s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:20:56 dist-test-slave-jq4t krb5kdc[18766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:20:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:20:56Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.229.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.229.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.229.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/rangeradmin_127.29.229.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.229.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/rangeradmin_127.29.229.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.229.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.229.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.229.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/rangerlookup_127.29.229.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.229.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/rangerlookup_127.29.229.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.229.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.229.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.229.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/HTTP_127.29.229.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.229.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/HTTP_127.29.229.20@KRBTEST.COM.keytab.
I20260811 22:20:56.396143 30612 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskD_Yjlj/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/postgres -l logfile start
2026-08-11 22:20:59.112 UTC [18825] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 22:20:59.112 UTC [18825] LOG: listening on IPv4 address "127.29.229.20", port 38557
2026-08-11 22:20:59.115 UTC [18825] LOG: listening on Unix socket "/tmp/.s.PGSQL.38557"
2026-08-11 22:20:59.142 UTC [18829] LOG: database system was shut down at 2026-08-11 22:20:57 UTC
2026-08-11 22:20:59.146 UTC [18825] LOG: database system is ready to accept connections
I20260811 22:21:00.414764 30612 mini_postgres.cc:96] Postgres bound to 38557
2026-08-11 22:21:00.446 UTC [18839] FATAL: database "slave" does not exist
127.29.229.20:38557 - accepting connections
I20260811 22:21:00.448678 30612 mini_ranger.cc:161] Starting Ranger...
I20260811 22:21:00.487510 30612 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 22:21:01.019419 30612 mini_ranger.cc:87] Created ranger Postgres database
I20260811 22:21:01.019650 30612 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ranger-admin
2026-08-11 22:21:01,364 [I] DB FLAVOR :POSTGRES
2026-08-11 22:21:01,365 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:21:01,365 [I] Checking connection..
2026-08-11 22:21:01,365 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 22:21:01,638 [I] Checking connection passed.
2026-08-11 22:21:01,639 [I] --------- Verifying version history table ---------
2026-08-11 22:21:01,639 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:21:01,924 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:21:01,924 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:21:02,178 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:21:02,178 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 22:21:02,178 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 22:21:02.463 UTC [18945] WARNING: there is no transaction in progress
2026-08-11 22:21:02,473 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 22:21:02,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:21:02,732 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 22:21:02,733 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:21:02,733 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 22:21:02,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 22:21:03,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-jq4t.c.gcp-upstream.internal','N') ;"
2026-08-11 22:21:03,530 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:21:03,800 [I] Table x_portal_user does not exist in database ranger
2026-08-11 22:21:03,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:21:04,058 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 22:21:04,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:21:04,324 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:21:04,605 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 22:21:04,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 22:21:05.708 UTC [19113] WARNING: there is no transaction in progress
2026-08-11 22:21:05.725 UTC [19113] WARNING: there is no transaction in progress
2026-08-11 22:21:05.744 UTC [19113] WARNING: there is no transaction in progress
2026-08-11 22:21:05.763 UTC [19113] WARNING: there is no transaction in progress
2026-08-11 22:21:05.782 UTC [19113] WARNING: there is no transaction in progress
2026-08-11 22:21:05.799 UTC [19113] WARNING: there is no transaction in progress
2026-08-11 22:21:05.862 UTC [19113] WARNING: there is no transaction in progress
2026-08-11 22:21:05.869 UTC [19113] WARNING: there is no transaction in progress
2026-08-11 22:21:05.884 UTC [19113] WARNING: there is no transaction in progress
2026-08-11 22:21:05.895 UTC [19113] WARNING: there is no transaction in progress
2026-08-11 22:21:05.906 UTC [19113] WARNING: there is no transaction in progress
2026-08-11 22:21:05.920 UTC [19113] WARNING: there is no transaction in progress
2026-08-11 22:21:05.930 UTC [19113] WARNING: there is no transaction in progress
2026-08-11 22:21:05.936 UTC [19113] WARNING: there is no transaction in progress
2026-08-11 22:21:05.944 UTC [19113] WARNING: there is no transaction in progress
2026-08-11 22:21:05.951 UTC [19113] WARNING: there is no transaction in progress
2026-08-11 22:21:06,722 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 22:21:06,723 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 22:21:06,995 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 22:21:06,996 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:21:07,289 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 22:21:07,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:21:07,572 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 22:21:07,573 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:21:07,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:21:08,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-jq4t.c.gcp-upstream.internal';"
2026-08-11 22:21:08,371 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 22:21:08,371 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 22:21:08,651 [I] DB_PATCHES have already been applied
I20260811 22:21:08.657771 30612 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ranger-admin:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskD_Yjlj/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 22:21:08.657897 30612 mini_ranger.cc:193] Using host: 127.29.229.20
I20260811 22:21:08.683442 30612 mini_ranger.cc:227] Ranger admin URL: http://127.29.229.20:52015
Aug 11, 2026 10:21:09 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:21:09 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:21:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 10:21:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 10:21:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 10:21:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 10:21:09 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 22:21:10.191880 30612 mini_ranger.cc:160] Time spent starting Ranger: real 9.743s user 0.001s sys 0.082s
Aug 11, 2026 10:21:32 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 10:21:32 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 10:21:32 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 10:21:32 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 10:21:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 10:21:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 10:21:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 10:21:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 10:21:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 10:21:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 10:21:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 10:21:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 10:21:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 10:21:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 10:21:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 10:21:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 10:21:32 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 10:21:33 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 22:21:34.009699 30612 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.229.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.229.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.229.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/rangerkms_127.29.229.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.229.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/rangerkms_127.29.229.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.229.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.229.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.229.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/HTTP_127.29.229.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.229.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/HTTP_127.29.229.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 22:21:34 dist-test-slave-jq4t krb5kdc[18766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486894, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 22:21:34.291721 30612 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 22:21:34.324906 30612 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 22:21:35.304737 30612 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 22:21:35.304999 30612 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ranger-kms
I20260811 22:21:35.305054 30612 mini_ranger_kms.cc:227] Using postgres at 127.29.229.20:38557
2026-08-11 22:21:35,744 [I] DB FLAVOR :POSTGRES
2026-08-11 22:21:35,745 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:21:35,745 [I] Checking connection
2026-08-11 22:21:35,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 22:21:36,037 [I] connection success
2026-08-11 22:21:36,037 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 22:21:36,037 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 22:21:36,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 22:21:36,333 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 22:21:36,333 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:21:36,333 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 22:21:36,334 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.229.20:38557/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 22:21:36,641 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 22:21:37.142866 30612 mini_ranger_kms.cc:310] Created kms service
I20260811 22:21:37.337711 30612 mini_ranger_kms.cc:326] Created kudu user
I20260811 22:21:37.387583 30612 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 22:21:37 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486894, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
I20260811 22:21:37.837566 30612 mini_ranger_kms.cc:384] Added ranger policy
I20260811 22:21:37.837980 30612 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ranger-kms:/tmp/dist-test-taskD_Yjlj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskD_Yjlj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 22:21:37.838058 30612 mini_ranger_kms.cc:240] Using host: 127.29.229.20
I20260811 22:21:37.865533 30612 mini_ranger_kms.cc:276] Ranger KMS PID: 19401
I20260811 22:21:37.865682 30612 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.229.20:50631
Aug 11, 2026 10:21:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:21:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:21:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 10:21:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 10:21:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 10:21:39 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 22:21:40 dist-test-slave-jq4t krb5kdc[18766](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486900, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.229.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 22:21:40 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486900, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.229.20@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
I20260811 22:21:40.847698 30612 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.556s user 0.082s sys 0.408s
I20260811 22:21:40.847952 30612 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 22:21:40.848121 30612 mini_ranger_kms.cc:401] 127.29.229.20:50631/kms/v1/keys
Aug 11 22:21:41 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486900, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.229.20@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
Aug 11 22:21:43 dist-test-slave-jq4t krb5kdc[18766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:21:43.709484 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:40235
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:42887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:40235
--ranger_config_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.229.20:50631/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:44.098814 19480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:44.099264 19480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:44.099434 19480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:44.107476 19480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:44.107647 19480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:44.107761 19480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:44.107877 19480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:44.107966 19480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:44.119307 19480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:42887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.229.20:50631/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:40235
--ranger_config_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:40235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:21:44.120842 19480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:44.122323 19480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:44.137483 19486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:44.137485 19487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:44.138345 19490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:44.138761 19480 server_base.cc:1034] running on GCE node
I20260811 22:21:44.139698 19480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:44.141458 19480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:44.142702 19480 hybrid_clock.cc:648] HybridClock initialized: now 1786486904142663 us; error 41 us; skew 500 ppm
I20260811 22:21:44.143098 19480 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:21:44 dist-test-slave-jq4t krb5kdc[18766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486904, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:44.152401 19480 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:21:44.155413 19480 webserver.cc:460] Webserver started at http://127.29.229.62:35767/ using document root <none> and password file <none>
I20260811 22:21:44.156569 19480 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:44.156755 19480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:44.157124 19480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:21:44 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486904, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
I20260811 22:21:44.227737 19480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "634f51bfe49f4ee9a7d939843910b8ba"
format_stamp: "Formatted at 2026-08-11 22:21:44 on dist-test-slave-jq4t"
server_key: "7d93724c0391322dc97be5854bf99730"
server_key_iv: "633b3a7e7be792dd524e68ea7a563763"
server_key_version: "kuduclusterkey@0"
I20260811 22:21:44.228979 19480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "634f51bfe49f4ee9a7d939843910b8ba"
format_stamp: "Formatted at 2026-08-11 22:21:44 on dist-test-slave-jq4t"
server_key: "7d93724c0391322dc97be5854bf99730"
server_key_iv: "633b3a7e7be792dd524e68ea7a563763"
server_key_version: "kuduclusterkey@0"
I20260811 22:21:44.236773 19480 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 22:21:44.258659 19507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:44.260749 19480 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:21:44.261004 19480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "634f51bfe49f4ee9a7d939843910b8ba"
format_stamp: "Formatted at 2026-08-11 22:21:44 on dist-test-slave-jq4t"
server_key: "7d93724c0391322dc97be5854bf99730"
server_key_iv: "633b3a7e7be792dd524e68ea7a563763"
server_key_version: "kuduclusterkey@0"
I20260811 22:21:44.261273 19480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:44.278704 19480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:44.283407 19480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:44.283766 19480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:44.307855 19480 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:40235
I20260811 22:21:44.307863 19578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:40235 every 8 connection(s)
I20260811 22:21:44.309384 19480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:44.310155 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 19480
I20260811 22:21:44.310405 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:21:44.311484 30612 external_mini_cluster.cc:1428] Setting key 57b9586629bb1807e351cfaf61d3bd1a
I20260811 22:21:44.314288 19580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:44.333464 19580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba: Bootstrap starting.
Aug 11 22:21:44 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:21:44.339123 19580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:44.341197 19580 log.cc:824] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:44.345538 19580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba: No bootstrap required, opened a new log
I20260811 22:21:44.350742 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:44.319765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41744 (local address 127.29.229.62:40235)
0811 22:21:44.321027 (+ 1262us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:44.321050 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:44.325236 (+ 4186us) server_negotiation.cc:407] Connection header received
0811 22:21:44.327185 (+ 1949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:44.327210 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:44.327655 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:44.328096 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:44.329631 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.331615 (+ 1984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.333409 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:44.334416 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:44.334634 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:44.339832 (+ 5198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:44.339843 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:44.339864 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:44.339932 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:44.344537 (+ 4605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.345480 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.345488 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.345494 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.345955 (+ 461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:44.346530 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:44.346545 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:44.346547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:44.347356 (+ 809us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:44.347796 (+ 440us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:44.348349 (+ 553us) server_negotiation.cc:299] Negotiation successful
0811 22:21:44.349032 (+ 683us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":909,"thread_start_us":502,"threads_started":1}
I20260811 22:21:44.358182 19580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634f51bfe49f4ee9a7d939843910b8ba" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40235 } }
I20260811 22:21:44.358726 19580 raft_consensus.cc:374] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:44.358850 19580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 634f51bfe49f4ee9a7d939843910b8ba, State: Initialized, Role: FOLLOWER
I20260811 22:21:44.359966 19580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634f51bfe49f4ee9a7d939843910b8ba" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40235 } }
I20260811 22:21:44.360271 19580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:44.360409 19580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:44.360615 19580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:44.363902 19580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634f51bfe49f4ee9a7d939843910b8ba" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40235 } }
I20260811 22:21:44.364614 19580 leader_election.cc:302] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 634f51bfe49f4ee9a7d939843910b8ba; no voters:
I20260811 22:21:44.365343 19580 leader_election.cc:288] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:21:44.365512 19591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:44.366362 19591 raft_consensus.cc:686] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [term 1 LEADER]: Becoming Leader. State: Replica: 634f51bfe49f4ee9a7d939843910b8ba, State: Running, Role: LEADER
I20260811 22:21:44.367069 19591 consensus_queue.cc:237] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634f51bfe49f4ee9a7d939843910b8ba" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40235 } }
I20260811 22:21:44.370317 19597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "634f51bfe49f4ee9a7d939843910b8ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634f51bfe49f4ee9a7d939843910b8ba" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40235 } } }
I20260811 22:21:44.370850 19597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:44.372660 19591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 634f51bfe49f4ee9a7d939843910b8ba. Latest consensus state: current_term: 1 leader_uuid: "634f51bfe49f4ee9a7d939843910b8ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634f51bfe49f4ee9a7d939843910b8ba" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40235 } } }
I20260811 22:21:44.373390 19591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:44.382303 19580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:44.382611 19601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:44.400393 19601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:44.411614 19601 catalog_manager.cc:1347] Generated new cluster ID: c1c72986c49e4d1f9eec8d56a8106d93
I20260811 22:21:44.411764 19601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:44.426669 19601 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:44.428808 19601 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:44.443364 19601 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba: Generated new TSK 0
I20260811 22:21:44.445194 19601 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:21:44.453712 19580 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
WARNING: no policy specified for kudu/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
W20260811 22:21:45.764329 30612 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.29.229.1: real 0.104s user 0.007s sys 0.035s
I20260811 22:21:45.766798 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.229.20:50631/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:40235
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:42887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:21:46 dist-test-slave-jq4t krb5kdc[18766](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486906, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 22:21:46.312747 19656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:46.313220 19656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:46.313414 19656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:46.325413 19656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:46.325628 19656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:46.325852 19656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:21:46.342832 19656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:42887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.229.20:50631/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:40235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:21:46.344990 19656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:46.348069 19656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:46.384022 19676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:46.384240 19675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:46.384907 19681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:46.385972 19656 server_base.cc:1034] running on GCE node
I20260811 22:21:46.386706 19656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:46.387845 19656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:46.389101 19656 hybrid_clock.cc:648] HybridClock initialized: now 1786486906389036 us; error 56 us; skew 500 ppm
I20260811 22:21:46.389469 19656 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:21:46 dist-test-slave-jq4t krb5kdc[18766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486906, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:46.392995 19656 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:21:46.394716 19656 webserver.cc:460] Webserver started at http://127.29.229.1:36423/ using document root <none> and password file <none>
I20260811 22:21:46.395522 19656 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:46.395687 19656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:46.396026 19656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:21:46 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486906, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
I20260811 22:21:46.426620 19656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "d28213ca759a4e6fb8e4581fb2739826"
format_stamp: "Formatted at 2026-08-11 22:21:46 on dist-test-slave-jq4t"
server_key: "26a3a403b02ab891eac5061b382513c6"
server_key_iv: "d1a2e4f2c45a0643c32d64e22bbc12de"
server_key_version: "kuduclusterkey@0"
I20260811 22:21:46.427798 19656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "d28213ca759a4e6fb8e4581fb2739826"
format_stamp: "Formatted at 2026-08-11 22:21:46 on dist-test-slave-jq4t"
server_key: "26a3a403b02ab891eac5061b382513c6"
server_key_iv: "d1a2e4f2c45a0643c32d64e22bbc12de"
server_key_version: "kuduclusterkey@0"
I20260811 22:21:46.449623 19656 fs_manager.cc:691] Time spent creating directory manager: real 0.021s user 0.011s sys 0.008s
I20260811 22:21:46.484236 19693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:46.486498 19656 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.009s sys 0.001s
I20260811 22:21:46.486829 19656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "d28213ca759a4e6fb8e4581fb2739826"
format_stamp: "Formatted at 2026-08-11 22:21:46 on dist-test-slave-jq4t"
server_key: "26a3a403b02ab891eac5061b382513c6"
server_key_iv: "d1a2e4f2c45a0643c32d64e22bbc12de"
server_key_version: "kuduclusterkey@0"
I20260811 22:21:46.487217 19656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:46.556313 19656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:46.562561 19656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:46.562947 19656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:46.564268 19656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:46.566360 19656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:46.566468 19656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:46.566668 19656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:46.566753 19656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:21:46 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486906, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
I20260811 22:21:46.954358 19859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:36731 every 8 connection(s)
I20260811 22:21:46.954774 19656 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:36731
I20260811 22:21:46.956663 19656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:21:46.966113 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 19656
I20260811 22:21:46.966414 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:21:46.967506 30612 external_mini_cluster.cc:1428] Setting key 0c898e299a0092bbc0ef2c31120f39ec
Aug 11 22:21:47 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486906, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:21:47.029407 19862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.965741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:34115 (local address 127.29.229.62:40235)
0811 22:21:46.992960 (+ 27219us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:46.992968 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:46.992999 (+ 31us) server_negotiation.cc:407] Connection header received
0811 22:21:46.993100 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:46.993106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:46.993209 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:46.993359 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:46.995886 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.997637 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:47.000278 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:47.000978 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:47.001111 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:47.006595 (+ 5484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:47.006606 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:47.006611 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:47.006660 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:47.010736 (+ 4076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:47.011971 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:47.011976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:47.011979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:47.012137 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:47.025388 (+ 13251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:47.025396 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:47.025400 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:47.025863 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:47.026065 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:47.028703 (+ 2638us) server_negotiation.cc:299] Negotiation successful
0811 22:21:47.029007 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27055,"thread_start_us":27721,"threads_started":1}
I20260811 22:21:47.030699 19861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:46.964948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:40235 (local address 127.29.229.1:34115)
0811 22:21:46.977371 (+ 12423us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:46.977453 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:46.979940 (+ 2487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:46.993795 (+ 13855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:46.993813 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:46.994595 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:46.995593 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:46.995624 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:46.997868 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:46.997874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:47.000049 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:47.000070 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:47.001521 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:47.001530 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:47.001980 (+ 450us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:47.002850 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:47.002893 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:47.006230 (+ 3337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:47.011011 (+ 4781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:47.011023 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:47.011047 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:47.011782 (+ 735us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:47.024637 (+ 12855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:47.024647 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:47.024654 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:47.025120 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:47.026162 (+ 1042us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:47.026174 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:47.026692 (+ 518us) client_negotiation.cc:769] Sending connection context
0811 22:21:47.027053 (+ 361us) client_negotiation.cc:240] Negotiation successful
0811 22:21:47.027670 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12013,"thread_start_us":16064,"threads_started":1}
I20260811 22:21:47.033342 19860 heartbeater.cc:347] Connected to a master server at 127.29.229.62:40235
I20260811 22:21:47.033727 19860 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:47.034771 19860 heartbeater.cc:511] Master 127.29.229.62:40235 requested a full tablet report, sending...
I20260811 22:21:47.038075 19527 ts_manager.cc:194] Registered new tserver with Master: d28213ca759a4e6fb8e4581fb2739826 (127.29.229.1:36731)
I20260811 22:21:47.040182 19527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:34115
WARNING: no policy specified for kudu/127.29.229.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.2@KRBTEST.COM" created.
Aug 11 22:21:47 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486906, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
W20260811 22:21:47.111999 30612 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.29.229.2: real 0.141s user 0.000s sys 0.030s
I20260811 22:21:47.113128 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.229.20:50631/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:40235
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:42887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:47.598331 19871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:47.598654 19871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:47.598821 19871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:47.606284 19871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:47.606438 19871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:47.606612 19871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:21:47.616636 19871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:42887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.229.20:50631/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:40235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:21:47.617980 19871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:47.619467 19871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:47.635543 19907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:47.635537 19906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:47.636772 19871 server_base.cc:1034] running on GCE node
W20260811 22:21:47.637122 19911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:47.637928 19871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:47.638989 19871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:47.640254 19871 hybrid_clock.cc:648] HybridClock initialized: now 1786486907640191 us; error 53 us; skew 500 ppm
I20260811 22:21:47.640720 19871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:21:47 dist-test-slave-jq4t krb5kdc[18766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486907, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:47.644531 19871 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:21:47.646293 19871 webserver.cc:460] Webserver started at http://127.29.229.2:34443/ using document root <none> and password file <none>
I20260811 22:21:47.647018 19871 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:47.647171 19871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:47.647486 19871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:21:47 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486907, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
I20260811 22:21:47.668923 19871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "88c06c89199146e58d856c4d76166c64"
format_stamp: "Formatted at 2026-08-11 22:21:47 on dist-test-slave-jq4t"
server_key: "36c587cb238b3a757923a5fc60f84ef9"
server_key_iv: "f1858e52af513273f28300bdf98892c6"
server_key_version: "kuduclusterkey@0"
I20260811 22:21:47.670267 19871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "88c06c89199146e58d856c4d76166c64"
format_stamp: "Formatted at 2026-08-11 22:21:47 on dist-test-slave-jq4t"
server_key: "36c587cb238b3a757923a5fc60f84ef9"
server_key_iv: "f1858e52af513273f28300bdf98892c6"
server_key_version: "kuduclusterkey@0"
I20260811 22:21:47.678752 19871 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260811 22:21:47.698441 19928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:47.700462 19871 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:21:47.700718 19871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "88c06c89199146e58d856c4d76166c64"
format_stamp: "Formatted at 2026-08-11 22:21:47 on dist-test-slave-jq4t"
server_key: "36c587cb238b3a757923a5fc60f84ef9"
server_key_iv: "f1858e52af513273f28300bdf98892c6"
server_key_version: "kuduclusterkey@0"
I20260811 22:21:47.700992 19871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:47.719393 19871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:47.724907 19871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:47.725281 19871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:47.726578 19871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:47.728648 19871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:47.728747 19871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:47.728925 19871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:47.728998 19871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:47.779181 19871 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:34245
I20260811 22:21:47.779224 20109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:34245 every 8 connection(s)
I20260811 22:21:47.780998 19871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:21:47.788651 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 19871
I20260811 22:21:47.788889 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:21:47.789636 30612 external_mini_cluster.cc:1428] Setting key 1cefade109a1105f53098fd64ad264d3
Aug 11 22:21:47 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486907, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:21:47.822746 20117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:47.783912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:45179 (local address 127.29.229.62:40235)
0811 22:21:47.796943 (+ 13031us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:47.796951 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:47.798896 (+ 1945us) server_negotiation.cc:407] Connection header received
0811 22:21:47.801602 (+ 2706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:47.801609 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:47.801706 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:47.801854 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:47.804057 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:47.805362 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:47.808133 (+ 2771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:47.808826 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:47.808960 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:47.813495 (+ 4535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:47.813503 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:47.813516 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:47.813566 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:47.817689 (+ 4123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:47.818977 (+ 1288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:47.818983 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:47.818987 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:47.819191 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:47.820012 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:47.820016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:47.820018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:47.820670 (+ 652us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:47.820863 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:47.821934 (+ 1071us) server_negotiation.cc:299] Negotiation successful
0811 22:21:47.822232 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12873,"thread_start_us":12711,"threads_started":1}
I20260811 22:21:47.823814 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:47.784665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:40235 (local address 127.29.229.2:45179)
0811 22:21:47.798277 (+ 13612us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:47.798335 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:47.801240 (+ 2905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:47.802166 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:47.802182 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:47.802871 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:47.803785 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:47.803814 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:47.805644 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:47.805650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:47.807936 (+ 2286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:47.807962 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:47.809066 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:47.809074 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:47.809372 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:47.810238 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:47.810268 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:47.813204 (+ 2936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:47.817879 (+ 4675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:47.817890 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:47.817911 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:47.818733 (+ 822us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:47.819567 (+ 834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:47.819572 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:47.819576 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:47.819874 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:47.820848 (+ 974us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:47.820858 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:47.821365 (+ 507us) client_negotiation.cc:769] Sending connection context
0811 22:21:47.821739 (+ 374us) client_negotiation.cc:240] Negotiation successful
0811 22:21:47.822398 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13213,"thread_start_us":12963,"threads_started":1}
I20260811 22:21:47.825593 20112 heartbeater.cc:347] Connected to a master server at 127.29.229.62:40235
I20260811 22:21:47.825950 20112 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:47.827071 20112 heartbeater.cc:511] Master 127.29.229.62:40235 requested a full tablet report, sending...
I20260811 22:21:47.829046 19527 ts_manager.cc:194] Registered new tserver with Master: 88c06c89199146e58d856c4d76166c64 (127.29.229.2:34245)
I20260811 22:21:47.830192 19527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:45179
WARNING: no policy specified for kudu/127.29.229.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
I20260811 22:21:47.881126 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.229.20:50631/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:40235
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:42887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:21:48.044384 19860 heartbeater.cc:503] Master 127.29.229.62:40235 was elected leader, sending a full tablet report...
W20260811 22:21:48.245505 20122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:48.245807 20122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:48.245901 20122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:48.254388 20122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:48.254520 20122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:48.254720 20122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:21:48.265754 20122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:42887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.229.20:50631/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:40235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:21:48.267166 20122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:48.268605 20122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:48.283056 20129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:48.283231 20133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:48.283623 20128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:48.283944 20122 server_base.cc:1034] running on GCE node
I20260811 22:21:48.284732 20122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:48.285914 20122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:48.287222 20122 hybrid_clock.cc:648] HybridClock initialized: now 1786486908287169 us; error 43 us; skew 500 ppm
I20260811 22:21:48.287590 20122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:21:48 dist-test-slave-jq4t krb5kdc[18766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486908, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:48.291019 20122 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:21:48.292660 20122 webserver.cc:460] Webserver started at http://127.29.229.3:46783/ using document root <none> and password file <none>
I20260811 22:21:48.293411 20122 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:48.293602 20122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:48.293924 20122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:21:48 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486908, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
I20260811 22:21:48.311173 20122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "507253424ae14ca8a5845f55cad381f0"
format_stamp: "Formatted at 2026-08-11 22:21:48 on dist-test-slave-jq4t"
server_key: "dfd6ef12e44437808887b258cb458233"
server_key_iv: "ea9f05195ee13687b305d759f2cbee14"
server_key_version: "kuduclusterkey@0"
I20260811 22:21:48.312251 20122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "507253424ae14ca8a5845f55cad381f0"
format_stamp: "Formatted at 2026-08-11 22:21:48 on dist-test-slave-jq4t"
server_key: "dfd6ef12e44437808887b258cb458233"
server_key_iv: "ea9f05195ee13687b305d759f2cbee14"
server_key_version: "kuduclusterkey@0"
I20260811 22:21:48.319726 20122 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260811 22:21:48.336997 20150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:48.338907 20122 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260811 22:21:48.339172 20122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "507253424ae14ca8a5845f55cad381f0"
format_stamp: "Formatted at 2026-08-11 22:21:48 on dist-test-slave-jq4t"
server_key: "dfd6ef12e44437808887b258cb458233"
server_key_iv: "ea9f05195ee13687b305d759f2cbee14"
server_key_version: "kuduclusterkey@0"
I20260811 22:21:48.339455 20122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:48.351545 20122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:48.355823 20122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:48.356143 20122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:48.357625 20122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:48.359633 20122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:48.359773 20122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:48.359930 20122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:48.360010 20122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:48.400882 20122 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:42487
I20260811 22:21:48.400885 20326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:42487 every 8 connection(s)
I20260811 22:21:48.402400 20122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:21:48.407850 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 20122
I20260811 22:21:48.408057 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:21:48.408630 30612 external_mini_cluster.cc:1428] Setting key f5fcc538ce6e1daaa2ad9872e16fa819
Aug 11 22:21:48 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486908, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:21:48.423801 20332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:48.405223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:42779 (local address 127.29.229.62:40235)
0811 22:21:48.405777 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:48.405781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:48.406802 (+ 1021us) server_negotiation.cc:407] Connection header received
0811 22:21:48.409063 (+ 2261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:48.409070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:48.409161 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:48.409284 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:48.410773 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.411571 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.413354 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.413780 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.413926 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:48.417558 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:48.417563 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:48.417566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:48.417607 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:48.420315 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.421147 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.421153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.421156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.421281 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.421748 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.421753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.421758 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.422057 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:48.422225 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:48.423078 (+ 853us) server_negotiation.cc:299] Negotiation successful
0811 22:21:48.423418 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":319,"threads_started":1}
I20260811 22:21:48.424986 20333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:48.405802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:40235 (local address 127.29.229.3:42779)
0811 22:21:48.406635 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:48.406684 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:48.408753 (+ 2069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:48.409544 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:48.409553 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:48.410003 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:48.410595 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:48.410613 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.411749 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.411753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:48.413179 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:48.413192 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.414052 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.414061 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:48.414311 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:48.415060 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:48.415085 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:48.417293 (+ 2208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:48.420502 (+ 3209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:48.420514 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:48.420532 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:48.420995 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:48.421396 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:48.421400 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:48.421404 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:48.421621 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:48.422191 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:48.422200 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:48.422667 (+ 467us) client_negotiation.cc:769] Sending connection context
0811 22:21:48.422976 (+ 309us) client_negotiation.cc:240] Negotiation successful
0811 22:21:48.423694 (+ 718us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":373,"threads_started":1}
I20260811 22:21:48.426419 20327 heartbeater.cc:347] Connected to a master server at 127.29.229.62:40235
I20260811 22:21:48.426698 20327 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:48.427479 20327 heartbeater.cc:511] Master 127.29.229.62:40235 requested a full tablet report, sending...
I20260811 22:21:48.429016 19527 ts_manager.cc:194] Registered new tserver with Master: 507253424ae14ca8a5845f55cad381f0 (127.29.229.3:42487)
I20260811 22:21:48.429836 19527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:42779
I20260811 22:21:48.437930 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:21:48.439687 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 19656
Aug 11 22:21:48 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:21:48.460448 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:48.440737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57116 (local address 127.29.229.1:36731)
0811 22:21:48.441379 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:48.441386 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:48.441413 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:21:48.441483 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:48.441489 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:48.441689 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:48.441830 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:48.443385 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.444752 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.446531 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.447121 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.447313 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:48.452791 (+ 5478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:48.452815 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:48.452836 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:48.452897 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:48.456700 (+ 3803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.457663 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.457673 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.457676 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.457896 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.458395 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.458400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.458403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.458957 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:48.459470 (+ 513us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:48.459816 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 22:21:48.460132 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":331,"threads_started":1}
I20260811 22:21:48.461953 19855 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:21:48.546900 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 19656
I20260811 22:21:48.578554 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 19871
Aug 11 22:21:48 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.2@KRBTEST.COM
I20260811 22:21:48.597282 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:48.579599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40610 (local address 127.29.229.2:34245)
0811 22:21:48.580153 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:48.580160 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:48.580186 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:21:48.580247 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:48.580253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:48.580459 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:48.580812 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:48.582152 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.583180 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.584568 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.585275 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.585427 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:48.590422 (+ 4995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:48.590434 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:48.590450 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:48.590500 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:48.594047 (+ 3547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.594902 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.594910 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.594912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.595066 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.595475 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.595482 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.595485 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.595854 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:48.596177 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:48.596579 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 22:21:48.596879 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":285,"threads_started":1}
I20260811 22:21:48.598601 20101 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:21:48.685371 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 19871
I20260811 22:21:48.711477 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 20122
Aug 11 22:21:48 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.3@KRBTEST.COM
I20260811 22:21:48.733444 20344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:48.712683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51020 (local address 127.29.229.3:42487)
0811 22:21:48.713744 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:48.713752 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:48.713783 (+ 31us) server_negotiation.cc:407] Connection header received
0811 22:21:48.713849 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:48.713856 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:48.714092 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:48.714275 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:48.716221 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.717585 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.719546 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:48.720377 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:48.720684 (+ 307us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:48.725538 (+ 4854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:48.725554 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:48.725571 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:48.725621 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:48.729341 (+ 3720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.730458 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.730470 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.730473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.730736 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:48.731474 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:48.731484 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:48.731489 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:48.731970 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:48.732341 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:48.732747 (+ 406us) server_negotiation.cc:299] Negotiation successful
0811 22:21:48.733056 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":904,"thread_start_us":542,"threads_started":1}
I20260811 22:21:48.734931 20318 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:21:48.821906 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 20122
I20260811 22:21:48.849021 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 19480
I20260811 22:21:48.850414 19574 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:21:48.961182 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 19480
I20260811 22:21:49.000871 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:40235
--webserver_interface=127.29.229.62
--webserver_port=35767
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:42887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:40235
--ranger_config_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.229.20:50631/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.229.20:50631/kms with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:49.363559 20351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:49.363881 20351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:49.364032 20351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:49.371671 20351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:49.371800 20351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:49.371892 20351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:49.371982 20351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:49.372082 20351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:49.385932 20351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:42887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.229.20:50631/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:40235
--ranger_config_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:40235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=35767
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:21:49.389365 20351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:49.391505 20351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:49.409056 20357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:49.408969 20361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:49.408944 20365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:49.410111 20351 server_base.cc:1034] running on GCE node
I20260811 22:21:49.411339 20351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:49.413183 20351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:49.414481 20351 hybrid_clock.cc:648] HybridClock initialized: now 1786486909414416 us; error 62 us; skew 500 ppm
I20260811 22:21:49.414963 20351 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:21:49 dist-test-slave-jq4t krb5kdc[18766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486909, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:49.424264 20351 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:21:49.426549 20351 webserver.cc:460] Webserver started at http://127.29.229.62:35767/ using document root <none> and password file <none>
I20260811 22:21:49.427460 20351 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:49.427587 20351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:21:49 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486909, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
I20260811 22:21:49.507933 20351 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:21:49.511190 20378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:49.512746 20351 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:21:49.512969 20351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "634f51bfe49f4ee9a7d939843910b8ba"
format_stamp: "Formatted at 2026-08-11 22:21:44 on dist-test-slave-jq4t"
server_key: "7d93724c0391322dc97be5854bf99730"
server_key_iv: "633b3a7e7be792dd524e68ea7a563763"
server_key_version: "kuduclusterkey@0"
I20260811 22:21:49.514003 20351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:49.530431 20351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:49.536031 20351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:49.536453 20351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:49.560324 20351 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:40235
I20260811 22:21:49.560282 20454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:40235 every 8 connection(s)
I20260811 22:21:49.561977 20351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:49.568233 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 20351
I20260811 22:21:49.569380 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.229.20:50631/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:36731
--local_ip_for_outbound_sockets=127.29.229.1
--tserver_master_addrs=127.29.229.62:40235
--webserver_port=36423
--webserver_interface=127.29.229.1
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:42887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.229.20:50631/kms with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:21:49.569900 20456 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:21:49.575301 20456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba: Bootstrap starting.
I20260811 22:21:49.598424 20456 log.cc:824] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:49.608337 20456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:21:49.608992 20456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba: Bootstrap complete.
I20260811 22:21:49.620888 20456 raft_consensus.cc:348] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634f51bfe49f4ee9a7d939843910b8ba" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40235 } }
I20260811 22:21:49.621794 20456 raft_consensus.cc:729] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 634f51bfe49f4ee9a7d939843910b8ba, State: Initialized, Role: FOLLOWER
I20260811 22:21:49.622476 20456 consensus_queue.cc:260] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634f51bfe49f4ee9a7d939843910b8ba" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40235 } }
I20260811 22:21:49.622715 20456 raft_consensus.cc:388] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:21:49.622833 20456 raft_consensus.cc:482] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:21:49.623032 20456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:21:49.627134 20456 raft_consensus.cc:504] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634f51bfe49f4ee9a7d939843910b8ba" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40235 } }
I20260811 22:21:49.627727 20456 leader_election.cc:302] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 634f51bfe49f4ee9a7d939843910b8ba; no voters:
I20260811 22:21:49.628394 20456 leader_election.cc:288] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:21:49.628607 20462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:21:49.629720 20462 raft_consensus.cc:686] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [term 2 LEADER]: Becoming Leader. State: Replica: 634f51bfe49f4ee9a7d939843910b8ba, State: Running, Role: LEADER
I20260811 22:21:49.630405 20462 consensus_queue.cc:237] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634f51bfe49f4ee9a7d939843910b8ba" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40235 } }
I20260811 22:21:49.630785 20456 sys_catalog.cc:564] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:49.633104 20467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 634f51bfe49f4ee9a7d939843910b8ba. Latest consensus state: current_term: 2 leader_uuid: "634f51bfe49f4ee9a7d939843910b8ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634f51bfe49f4ee9a7d939843910b8ba" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40235 } } }
I20260811 22:21:49.633553 20467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:49.634200 20463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "634f51bfe49f4ee9a7d939843910b8ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634f51bfe49f4ee9a7d939843910b8ba" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40235 } } }
I20260811 22:21:49.634533 20463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:49.638244 20473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:49.646145 20473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:49.648027 20473 catalog_manager.cc:1259] Loaded cluster ID: c1c72986c49e4d1f9eec8d56a8106d93
I20260811 22:21:49.648141 20473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:21:49.652290 20473 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:49.653987 20473 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba: Loaded TSK: 0
I20260811 22:21:49.655651 20473 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:21:49.663874 20456 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260811 22:21:50.076381 20458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:50.076990 20458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:50.077216 20458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:50.090008 20458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:50.090260 20458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:50.090548 20458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:21:50.107771 20458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:42887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.229.20:50631/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:36731
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=36423
--tserver_master_addrs=127.29.229.62:40235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:21:50.110651 20458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:50.113276 20458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:50.143625 20515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:50.143651 20518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:50.148799 20520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:50.150028 20458 server_base.cc:1034] running on GCE node
I20260811 22:21:50.151113 20458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:50.153363 20458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:50.159989 20458 hybrid_clock.cc:648] HybridClock initialized: now 1786486910158937 us; error 1062 us; skew 500 ppm
I20260811 22:21:50.160605 20458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:21:50 dist-test-slave-jq4t krb5kdc[18766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486910, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:50.165810 20458 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:21:50.170083 20458 webserver.cc:460] Webserver started at http://127.29.229.1:36423/ using document root <none> and password file <none>
I20260811 22:21:50.171082 20458 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:50.171252 20458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:21:50 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486910, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
I20260811 22:21:50.203349 20458 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.000s sys 0.008s
I20260811 22:21:50.207719 20531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:50.209710 20458 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260811 22:21:50.209903 20458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "d28213ca759a4e6fb8e4581fb2739826"
format_stamp: "Formatted at 2026-08-11 22:21:46 on dist-test-slave-jq4t"
server_key: "26a3a403b02ab891eac5061b382513c6"
server_key_iv: "d1a2e4f2c45a0643c32d64e22bbc12de"
server_key_version: "kuduclusterkey@0"
I20260811 22:21:50.211412 20458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:50.231261 20458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:50.237116 20458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:50.237473 20458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:50.240614 20458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:50.250267 20458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:50.250422 20458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:50.250644 20458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:50.250820 20458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:50.345716 20458 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:36731
I20260811 22:21:50.345767 20716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:36731 every 8 connection(s)
I20260811 22:21:50.347523 20458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:21:50.353351 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 20458
I20260811 22:21:50.354224 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.229.20:50631/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:34245
--local_ip_for_outbound_sockets=127.29.229.2
--tserver_master_addrs=127.29.229.62:40235
--webserver_port=34443
--webserver_interface=127.29.229.2
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:42887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.229.20:50631/kms with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:21:50 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486910, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:21:50.384457 20721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:50.351679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:40235 (local address 127.29.229.1:54181)
0811 22:21:50.352843 (+ 1164us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:50.352898 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:50.355605 (+ 2707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:50.358133 (+ 2528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:50.358147 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:50.358843 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:50.359686 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:50.359716 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:50.362302 (+ 2586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:50.362310 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:50.364457 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:50.364511 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:50.366466 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:50.366477 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:50.366793 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:50.367684 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:50.367730 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:50.373294 (+ 5564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:50.378881 (+ 5587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:50.378893 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:50.378919 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:50.379607 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:50.380161 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:50.380173 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:50.380181 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:50.380399 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:50.381380 (+ 981us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:50.381389 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:50.381863 (+ 474us) client_negotiation.cc:769] Sending connection context
0811 22:21:50.382208 (+ 345us) client_negotiation.cc:240] Negotiation successful
0811 22:21:50.382876 (+ 668us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":857,"thread_start_us":624,"threads_started":1}
I20260811 22:21:50.387465 20720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:50.351612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:54181 (local address 127.29.229.62:40235)
0811 22:21:50.352401 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:50.352433 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:50.356014 (+ 3581us) server_negotiation.cc:407] Connection header received
0811 22:21:50.356465 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:50.356512 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:50.357001 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:50.357392 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:50.359996 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:50.362085 (+ 2089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:50.364670 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:50.365794 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:50.365993 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:50.373684 (+ 7691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:50.373693 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:50.373711 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:50.373763 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:50.378665 (+ 4902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:50.379796 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:50.379813 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:50.379819 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:50.380011 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:50.380558 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:50.380563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:50.380567 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:50.381209 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:50.381467 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:50.385390 (+ 3923us) server_negotiation.cc:299] Negotiation successful
0811 22:21:50.385905 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":300,"threads_started":1}
I20260811 22:21:50.391348 20717 heartbeater.cc:347] Connected to a master server at 127.29.229.62:40235
I20260811 22:21:50.391784 20717 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:50.392917 20717 heartbeater.cc:511] Master 127.29.229.62:40235 requested a full tablet report, sending...
W20260811 22:21:50.397593 20717 heartbeater.cc:644] Failed to heartbeat to 127.29.229.62:40235 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 22:21:50.397881 20717 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:50.398309 20717 heartbeater.cc:511] Master 127.29.229.62:40235 requested a full tablet report, sending...
I20260811 22:21:50.399847 20717 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:50.400207 20717 heartbeater.cc:511] Master 127.29.229.62:40235 requested a full tablet report, sending...
W20260811 22:21:50.401603 20717 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:21:50.845921 20722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:50.846287 20722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:50.846457 20722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:50.856096 20722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:50.856354 20722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:50.856611 20722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:21:50.873785 20722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:42887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.229.20:50631/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:34245
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=34443
--tserver_master_addrs=127.29.229.62:40235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:21:50.875737 20722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:50.877641 20722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:50.898233 20750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:50.906096 20749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:50.916671 20754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:50.919135 20722 server_base.cc:1034] running on GCE node
I20260811 22:21:50.920307 20722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:50.923144 20722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:50.924551 20722 hybrid_clock.cc:648] HybridClock initialized: now 1786486910924439 us; error 73 us; skew 500 ppm
I20260811 22:21:50.925055 20722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:21:50 dist-test-slave-jq4t krb5kdc[18766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486910, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:50.930461 20722 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:21:50.934103 20722 webserver.cc:460] Webserver started at http://127.29.229.2:34443/ using document root <none> and password file <none>
I20260811 22:21:50.935384 20722 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:50.935604 20722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:21:50 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486910, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
I20260811 22:21:50.964179 20722 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.006s sys 0.000s
I20260811 22:21:50.968101 20766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:50.970088 20722 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:21:50.970309 20722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "88c06c89199146e58d856c4d76166c64"
format_stamp: "Formatted at 2026-08-11 22:21:47 on dist-test-slave-jq4t"
server_key: "36c587cb238b3a757923a5fc60f84ef9"
server_key_iv: "f1858e52af513273f28300bdf98892c6"
server_key_version: "kuduclusterkey@0"
I20260811 22:21:50.971796 20722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 11 22:21:50 dist-test-slave-jq4t krb5kdc[18766](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486910, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:51.004702 20722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:51.010345 20722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:51.010766 20722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:51.013065 20722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:51.014936 20722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:51.015053 20722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:51.015164 20722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:51.015245 20722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:51.075995 20722 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:34245
I20260811 22:21:51.076126 20929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:34245 every 8 connection(s)
I20260811 22:21:51.078286 20722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:21:51.088580 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 20722
I20260811 22:21:51.089466 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.229.20:50631/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:42487
--local_ip_for_outbound_sockets=127.29.229.3
--tserver_master_addrs=127.29.229.62:40235
--webserver_port=46783
--webserver_interface=127.29.229.3
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:42887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.229.20:50631/kms with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:21:51 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486910, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:21:51.124333 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:51.081527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:50083 (local address 127.29.229.62:40235)
0811 22:21:51.096860 (+ 15333us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:51.096868 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:51.100543 (+ 3675us) server_negotiation.cc:407] Connection header received
0811 22:21:51.102393 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:51.102401 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:51.102517 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:51.102614 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:51.105216 (+ 2602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.106418 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.109086 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.109794 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.109916 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:51.115318 (+ 5402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:51.115341 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:51.115346 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:51.115396 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:51.119728 (+ 4332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:51.121057 (+ 1329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:51.121063 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:51.121067 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:51.121240 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:51.121866 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:51.121872 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:51.121876 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:51.122266 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:51.122435 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:51.123614 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0811 22:21:51.123901 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15166,"thread_start_us":15364,"threads_started":1}
I20260811 22:21:51.127135 20933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:51.082223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:40235 (local address 127.29.229.2:50083)
0811 22:21:51.098006 (+ 15783us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:51.098089 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:51.101926 (+ 3837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:51.103250 (+ 1324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:51.103263 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:51.104024 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:51.104951 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:51.104993 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.106614 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.106620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:51.108865 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:51.108883 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.110356 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.110365 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:51.110696 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:51.111664 (+ 968us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:51.111697 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:51.114933 (+ 3236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:51.120073 (+ 5140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:51.120085 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:51.120106 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:51.120878 (+ 772us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:51.121416 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:51.121420 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:51.121424 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:51.121711 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:51.122544 (+ 833us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:51.122555 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:51.123091 (+ 536us) client_negotiation.cc:769] Sending connection context
0811 22:21:51.124910 (+ 1819us) client_negotiation.cc:240] Negotiation successful
0811 22:21:51.125575 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15042,"thread_start_us":21218,"threads_started":1}
I20260811 22:21:51.129364 20930 heartbeater.cc:347] Connected to a master server at 127.29.229.62:40235
I20260811 22:21:51.129732 20930 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:51.131026 20930 heartbeater.cc:511] Master 127.29.229.62:40235 requested a full tablet report, sending...
I20260811 22:21:51.136011 20404 ts_manager.cc:194] Registered new tserver with Master: 88c06c89199146e58d856c4d76166c64 (127.29.229.2:34245)
I20260811 22:21:51.138666 20404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:50083
I20260811 22:21:51.401851 20717 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:51.402654 20717 heartbeater.cc:511] Master 127.29.229.62:40235 requested a full tablet report, sending...
I20260811 22:21:51.404436 20404 ts_manager.cc:194] Registered new tserver with Master: d28213ca759a4e6fb8e4581fb2739826 (127.29.229.1:36731)
I20260811 22:21:51.405679 20404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:54181
Aug 11 22:21:51 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486910, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
W20260811 22:21:51.581907 20937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:51.582254 20937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:51.582458 20937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:51.590117 20937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:51.590281 20937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:51.590485 20937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:21:51.600929 20937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:42887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.229.20:50631/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:42487
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=46783
--tserver_master_addrs=127.29.229.62:40235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:21:51.602483 20937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:51.603832 20937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:51.620990 20944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:51.627912 20947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:51.627938 20937 server_base.cc:1034] running on GCE node
W20260811 22:21:51.628669 20951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:51.629487 20937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:51.630685 20937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:51.631979 20937 hybrid_clock.cc:648] HybridClock initialized: now 1786486911631918 us; error 58 us; skew 500 ppm
I20260811 22:21:51.632390 20937 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:21:51 dist-test-slave-jq4t krb5kdc[18766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486911, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:51.635754 20937 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:21:51.639377 20937 webserver.cc:460] Webserver started at http://127.29.229.3:46783/ using document root <none> and password file <none>
I20260811 22:21:51.640182 20937 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:51.640363 20937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:21:51 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486911, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
I20260811 22:21:51.662626 20937 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 22:21:51.665764 20963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:51.667346 20937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:21:51.667538 20937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "507253424ae14ca8a5845f55cad381f0"
format_stamp: "Formatted at 2026-08-11 22:21:48 on dist-test-slave-jq4t"
server_key: "dfd6ef12e44437808887b258cb458233"
server_key_iv: "ea9f05195ee13687b305d759f2cbee14"
server_key_version: "kuduclusterkey@0"
I20260811 22:21:51.668661 20937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:51.694880 20937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:51.699391 20937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:51.699723 20937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:51.701046 20937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:21:51.702674 20937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:21:51.702806 20937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:51.702950 20937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:21:51.703049 20937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:51.761919 21143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:42487 every 8 connection(s)
I20260811 22:21:51.762035 20937 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:42487
I20260811 22:21:51.763600 20937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:21:51.772181 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 20937
Aug 11 22:21:51 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486911, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:21:51.802867 21150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:51.768669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:44433 (local address 127.29.229.62:40235)
0811 22:21:51.777324 (+ 8655us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:51.777334 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:51.777366 (+ 32us) server_negotiation.cc:407] Connection header received
0811 22:21:51.777482 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:51.777488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:51.778592 (+ 1104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:51.778772 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:51.782314 (+ 3542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.783577 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.786031 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.786948 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.787097 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:51.793247 (+ 6150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:51.793254 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:51.793259 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:51.793318 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:51.798491 (+ 5173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:51.799641 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:51.799647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:51.799651 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:51.799814 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:51.800410 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:51.800415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:51.800419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:51.800802 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:51.800971 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:51.802213 (+ 1242us) server_negotiation.cc:299] Negotiation successful
0811 22:21:51.802486 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7721,"thread_start_us":7578,"threads_started":1}
I20260811 22:21:51.804288 21148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:51.767890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:40235 (local address 127.29.229.3:44433)
0811 22:21:51.770331 (+ 2441us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:51.770388 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:51.776078 (+ 5690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:51.779091 (+ 3013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:51.779108 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:51.779945 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:51.782020 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:51.782056 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.783788 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.783796 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:51.785702 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:51.785719 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.787434 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.787450 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:51.787795 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:51.790115 (+ 2320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:51.790151 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:51.792928 (+ 2777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:51.798714 (+ 5786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:51.798726 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:51.798747 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:51.799470 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:51.799971 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:51.799976 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:51.799980 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:51.800249 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:51.801050 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:51.801059 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:51.801539 (+ 480us) client_negotiation.cc:769] Sending connection context
0811 22:21:51.801882 (+ 343us) client_negotiation.cc:240] Negotiation successful
0811 22:21:51.802617 (+ 735us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2135,"thread_start_us":1873,"threads_started":1}
I20260811 22:21:51.806689 21144 heartbeater.cc:347] Connected to a master server at 127.29.229.62:40235
I20260811 22:21:51.807039 21144 heartbeater.cc:464] Registering TS with master...
I20260811 22:21:51.807958 21144 heartbeater.cc:511] Master 127.29.229.62:40235 requested a full tablet report, sending...
I20260811 22:21:51.809816 20404 ts_manager.cc:194] Registered new tserver with Master: 507253424ae14ca8a5845f55cad381f0 (127.29.229.3:42487)
I20260811 22:21:51.810995 20404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:44433
I20260811 22:21:51.821331 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:51.774921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39716 (local address 127.29.229.62:40235)
0811 22:21:51.783447 (+ 8526us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:51.783454 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:51.783472 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:21:51.783534 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:51.783539 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:51.783627 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:51.783726 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:51.788609 (+ 4883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.789813 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.795522 (+ 5709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.796184 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.796334 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:51.812092 (+ 15758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:51.812099 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:51.812104 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:51.812147 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:51.816409 (+ 4262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:51.817374 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:51.817380 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:51.817383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:51.817561 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:51.818574 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:51.818579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:51.818582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:51.819022 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:51.819377 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:51.820699 (+ 1322us) server_negotiation.cc:299] Negotiation successful
0811 22:21:51.820940 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8393,"thread_start_us":9006,"threads_started":1}
I20260811 22:21:51.824352 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:21:51 dist-test-slave-jq4t krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486910, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for HTTP/127.29.229.20@KRBTEST.COM
I20260811 22:21:51.864863 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:51.835427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39718 (local address 127.29.229.62:40235)
0811 22:21:51.835654 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:51.835660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:51.841507 (+ 5847us) server_negotiation.cc:407] Connection header received
0811 22:21:51.841843 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:51.841850 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:51.841964 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:51.842081 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:51.844202 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.845333 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.847390 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.848041 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.848178 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:51.853516 (+ 5338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:51.853523 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:51.853527 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:51.853576 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:51.857430 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:51.858826 (+ 1396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:51.858832 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:51.858835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:51.859019 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:51.861573 (+ 2554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:51.861579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:51.861583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:51.862308 (+ 725us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:51.862685 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:51.863793 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0811 22:21:51.864321 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:21:51.871157 20404 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:21:51.881289 20404 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:21:51.931643 21179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:51.907225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:36731 (local address 127.0.0.1:57126)
0811 22:21:51.911182 (+ 3957us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:51.911228 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:51.911455 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:51.919859 (+ 8404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:51.919868 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:51.919906 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:51.920368 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:51.920385 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.924231 (+ 3846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.924237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:51.926861 (+ 2624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:51.926881 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.930572 (+ 3691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.930580 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:51.930844 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:51.930918 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 22:21:51.931039 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 22:21:51.931250 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3641,"thread_start_us":22843,"threads_started":1}
I20260811 22:21:51.933094 21177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:51.907449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57126 (local address 127.29.229.1:36731)
0811 22:21:51.918973 (+ 11524us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:51.918981 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:51.919015 (+ 34us) server_negotiation.cc:407] Connection header received
0811 22:21:51.919231 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:51.919240 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:51.919554 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:51.920660 (+ 1106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:51.920918 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.924053 (+ 3135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.927176 (+ 3123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.930282 (+ 3106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.932181 (+ 1899us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:51.932451 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:51.932579 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 22:21:51.932771 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11333,"thread_start_us":11456,"threads_started":1}
I20260811 22:21:51.939355 20613 tablet_service.cc:1467] Processing CreateTablet for tablet 408c91b893b64ddf8ed0da0bdfb90b41 (DEFAULT_TABLE table=test-table [id=7ac24e2243b94c76b73ec18205461959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:51.942446 20613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 408c91b893b64ddf8ed0da0bdfb90b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:51.947566 21179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:51.933964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:42487 (local address 127.0.0.1:51028)
0811 22:21:51.934100 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:51.934118 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:51.934234 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:51.940604 (+ 6370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:51.940610 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:51.940642 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:51.940856 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:51.940870 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.942989 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.942995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:51.944869 (+ 1874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:51.944886 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.946834 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.946841 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:51.947044 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:51.947083 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 22:21:51.947160 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 22:21:51.947305 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 22:21:51.948764 21183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:51.930925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51028 (local address 127.29.229.3:42487)
0811 22:21:51.933977 (+ 3052us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:51.933986 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:51.937415 (+ 3429us) server_negotiation.cc:407] Connection header received
0811 22:21:51.937546 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:51.937554 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:51.937862 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:51.938085 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:51.941114 (+ 3029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.942809 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.945116 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.946641 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.947801 (+ 1160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:51.948055 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:51.948155 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 22:21:51.948351 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2866,"thread_start_us":2675,"threads_started":1}
I20260811 22:21:51.954401 21046 tablet_service.cc:1467] Processing CreateTablet for tablet 408c91b893b64ddf8ed0da0bdfb90b41 (DEFAULT_TABLE table=test-table [id=7ac24e2243b94c76b73ec18205461959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:51.958091 21046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 408c91b893b64ddf8ed0da0bdfb90b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:51.963486 21175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:51.910306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:34245 (local address 127.0.0.1:40626)
0811 22:21:51.929887 (+ 19581us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:51.929914 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:51.938478 (+ 8564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:51.947920 (+ 9442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:51.947925 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:51.947955 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:51.948315 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:51.948329 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.951700 (+ 3371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.951705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:51.953970 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:51.953989 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.962603 (+ 8614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.962612 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:51.962820 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:51.962860 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 22:21:51.962972 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 22:21:51.963130 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19438,"thread_start_us":984,"threads_started":1}
I20260811 22:21:51.964531 21181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:51.911071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40626 (local address 127.29.229.2:34245)
0811 22:21:51.918472 (+ 7401us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:51.918482 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:51.930043 (+ 11561us) server_negotiation.cc:407] Connection header received
0811 22:21:51.940677 (+ 10634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:51.940688 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:51.940971 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:51.941179 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:51.948573 (+ 7394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.950157 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.955239 (+ 5082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:51.956693 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:51.956836 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:51.957031 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:51.963972 (+ 6941us) server_negotiation.cc:299] Negotiation successful
0811 22:21:51.964179 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7195,"thread_start_us":5787,"threads_started":1}
I20260811 22:21:51.969255 20839 tablet_service.cc:1467] Processing CreateTablet for tablet 408c91b893b64ddf8ed0da0bdfb90b41 (DEFAULT_TABLE table=test-table [id=7ac24e2243b94c76b73ec18205461959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:21:51.972496 20839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 408c91b893b64ddf8ed0da0bdfb90b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:51.984112 21184 tablet_bootstrap.cc:492] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0: Bootstrap starting.
I20260811 22:21:51.990300 21184 tablet_bootstrap.cc:654] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:51.991782 21184 log.cc:824] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:51.995867 21187 tablet_bootstrap.cc:492] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826: Bootstrap starting.
I20260811 22:21:52.001955 21184 tablet_bootstrap.cc:492] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0: No bootstrap required, opened a new log
I20260811 22:21:52.002420 21184 ts_tablet_manager.cc:1397] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.004s
I20260811 22:21:52.003259 21187 tablet_bootstrap.cc:654] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:52.004745 21187 log.cc:824] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:52.013376 21191 tablet_bootstrap.cc:492] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64: Bootstrap starting.
I20260811 22:21:52.014652 21184 raft_consensus.cc:348] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d28213ca759a4e6fb8e4581fb2739826" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36731 } } peers { permanent_uuid: "88c06c89199146e58d856c4d76166c64" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 34245 } } peers { permanent_uuid: "507253424ae14ca8a5845f55cad381f0" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42487 } }
I20260811 22:21:52.015280 21184 raft_consensus.cc:374] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:52.015429 21184 raft_consensus.cc:729] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 507253424ae14ca8a5845f55cad381f0, State: Initialized, Role: FOLLOWER
I20260811 22:21:52.016281 21184 consensus_queue.cc:260] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d28213ca759a4e6fb8e4581fb2739826" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36731 } } peers { permanent_uuid: "88c06c89199146e58d856c4d76166c64" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 34245 } } peers { permanent_uuid: "507253424ae14ca8a5845f55cad381f0" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42487 } }
I20260811 22:21:52.019035 21187 tablet_bootstrap.cc:492] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826: No bootstrap required, opened a new log
I20260811 22:21:52.020781 21187 ts_tablet_manager.cc:1397] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.011s
I20260811 22:21:52.024732 21184 ts_tablet_manager.cc:1428] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0: Time spent starting tablet: real 0.022s user 0.013s sys 0.007s
I20260811 22:21:52.025203 21144 heartbeater.cc:503] Master 127.29.229.62:40235 was elected leader, sending a full tablet report...
I20260811 22:21:52.026539 21191 tablet_bootstrap.cc:654] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:52.028017 21191 log.cc:824] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64: Log is configured to *not* fsync() on all Append() calls
W20260811 22:21:52.031903 21146 tablet.cc:2369] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:21:52.038884 21187 raft_consensus.cc:348] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d28213ca759a4e6fb8e4581fb2739826" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36731 } } peers { permanent_uuid: "88c06c89199146e58d856c4d76166c64" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 34245 } } peers { permanent_uuid: "507253424ae14ca8a5845f55cad381f0" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42487 } }
I20260811 22:21:52.039546 21187 raft_consensus.cc:374] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:52.039690 21187 raft_consensus.cc:729] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d28213ca759a4e6fb8e4581fb2739826, State: Initialized, Role: FOLLOWER
I20260811 22:21:52.040336 21191 tablet_bootstrap.cc:492] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64: No bootstrap required, opened a new log
I20260811 22:21:52.040583 21187 consensus_queue.cc:260] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d28213ca759a4e6fb8e4581fb2739826" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36731 } } peers { permanent_uuid: "88c06c89199146e58d856c4d76166c64" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 34245 } } peers { permanent_uuid: "507253424ae14ca8a5845f55cad381f0" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42487 } }
I20260811 22:21:52.040819 21191 ts_tablet_manager.cc:1397] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64: Time spent bootstrapping tablet: real 0.028s user 0.010s sys 0.000s
I20260811 22:21:52.049518 21187 ts_tablet_manager.cc:1428] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826: Time spent starting tablet: real 0.029s user 0.012s sys 0.009s
I20260811 22:21:52.050071 20717 heartbeater.cc:503] Master 127.29.229.62:40235 was elected leader, sending a full tablet report...
I20260811 22:21:52.052719 21191 raft_consensus.cc:348] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d28213ca759a4e6fb8e4581fb2739826" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36731 } } peers { permanent_uuid: "88c06c89199146e58d856c4d76166c64" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 34245 } } peers { permanent_uuid: "507253424ae14ca8a5845f55cad381f0" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42487 } }
I20260811 22:21:52.053359 21191 raft_consensus.cc:374] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:52.053537 21191 raft_consensus.cc:729] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88c06c89199146e58d856c4d76166c64, State: Initialized, Role: FOLLOWER
I20260811 22:21:52.054903 21191 consensus_queue.cc:260] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d28213ca759a4e6fb8e4581fb2739826" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36731 } } peers { permanent_uuid: "88c06c89199146e58d856c4d76166c64" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 34245 } } peers { permanent_uuid: "507253424ae14ca8a5845f55cad381f0" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42487 } }
I20260811 22:21:52.064015 21191 ts_tablet_manager.cc:1428] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64: Time spent starting tablet: real 0.023s user 0.003s sys 0.017s
I20260811 22:21:52.064671 20930 heartbeater.cc:503] Master 127.29.229.62:40235 was elected leader, sending a full tablet report...
I20260811 22:21:52.083174 21197 raft_consensus.cc:482] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:52.083602 21197 raft_consensus.cc:504] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d28213ca759a4e6fb8e4581fb2739826" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36731 } } peers { permanent_uuid: "88c06c89199146e58d856c4d76166c64" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 34245 } } peers { permanent_uuid: "507253424ae14ca8a5845f55cad381f0" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42487 } }
I20260811 22:21:52.092897 21197 leader_election.cc:288] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d28213ca759a4e6fb8e4581fb2739826 (127.29.229.1:36731), 507253424ae14ca8a5845f55cad381f0 (127.29.229.3:42487)
I20260811 22:21:52.099536 21177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:52.086550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:42859 (local address 127.29.229.1:36731)
0811 22:21:52.086767 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:52.086773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:52.091732 (+ 4959us) server_negotiation.cc:407] Connection header received
0811 22:21:52.092816 (+ 1084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:52.092823 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:52.092908 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:52.092995 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:52.093837 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:52.094967 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:52.097168 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:52.098320 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:52.098489 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:52.098566 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:52.099147 (+ 581us) server_negotiation.cc:299] Negotiation successful
0811 22:21:52.099286 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:21:52.100260 21198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:52.086094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:36731 (local address 127.29.229.2:42859)
0811 22:21:52.091593 (+ 5499us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:52.091619 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:52.092601 (+ 982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:52.093268 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:52.093272 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:52.093311 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:52.093655 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:52.093665 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:52.095173 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:52.095179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:52.096995 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:52.097009 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:52.098738 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:52.098745 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:52.098932 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:52.098966 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 22:21:52.099737 (+ 771us) client_negotiation.cc:240] Negotiation successful
0811 22:21:52.099881 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5354,"thread_start_us":5163,"threads_started":1}
I20260811 22:21:52.101675 20650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "408c91b893b64ddf8ed0da0bdfb90b41" candidate_uuid: "88c06c89199146e58d856c4d76166c64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d28213ca759a4e6fb8e4581fb2739826" is_pre_election: true
I20260811 22:21:52.102290 20650 raft_consensus.cc:2454] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88c06c89199146e58d856c4d76166c64 in term 0.
W20260811 22:21:52.103188 20931 tablet.cc:2369] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:21:52.104156 20719 tablet.cc:2369] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:21:52.105118 20768 leader_election.cc:302] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88c06c89199146e58d856c4d76166c64, d28213ca759a4e6fb8e4581fb2739826; no voters:
I20260811 22:21:52.105687 21197 raft_consensus.cc:2781] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:52.105851 21197 raft_consensus.cc:482] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:52.105983 21197 raft_consensus.cc:3037] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:52.109405 21197 raft_consensus.cc:504] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d28213ca759a4e6fb8e4581fb2739826" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36731 } } peers { permanent_uuid: "88c06c89199146e58d856c4d76166c64" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 34245 } } peers { permanent_uuid: "507253424ae14ca8a5845f55cad381f0" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42487 } }
I20260811 22:21:52.110301 21197 leader_election.cc:288] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [CANDIDATE]: Term 1 election: Requested vote from peers d28213ca759a4e6fb8e4581fb2739826 (127.29.229.1:36731), 507253424ae14ca8a5845f55cad381f0 (127.29.229.3:42487)
I20260811 22:21:52.111135 20650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "408c91b893b64ddf8ed0da0bdfb90b41" candidate_uuid: "88c06c89199146e58d856c4d76166c64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d28213ca759a4e6fb8e4581fb2739826"
I20260811 22:21:52.111378 20650 raft_consensus.cc:3037] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:52.113238 21183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:52.092086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:32879 (local address 127.29.229.3:42487)
0811 22:21:52.092300 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:52.092307 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:52.100168 (+ 7861us) server_negotiation.cc:407] Connection header received
0811 22:21:52.100526 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:52.100532 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:52.100619 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:52.100690 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:21:52.101711 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:52.102824 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:52.110620 (+ 7796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:52.111931 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:52.112093 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:52.112173 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:52.112812 (+ 639us) server_negotiation.cc:299] Negotiation successful
0811 22:21:52.112964 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:21:52.113910 21199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:52.092267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:42487 (local address 127.29.229.2:32879)
0811 22:21:52.099577 (+ 7310us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:52.099599 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:52.100288 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:52.101173 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:52.101178 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:52.101203 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:21:52.101526 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:52.101537 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:52.105019 (+ 3482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:52.105024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:52.110368 (+ 5344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:52.110388 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:52.112382 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:52.112389 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:52.112609 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:52.112645 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:21:52.113417 (+ 772us) client_negotiation.cc:240] Negotiation successful
0811 22:21:52.113574 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7174,"thread_start_us":7625,"threads_started":1}
I20260811 22:21:52.115582 21071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "408c91b893b64ddf8ed0da0bdfb90b41" candidate_uuid: "88c06c89199146e58d856c4d76166c64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "507253424ae14ca8a5845f55cad381f0" is_pre_election: true
I20260811 22:21:52.116216 21071 raft_consensus.cc:2454] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88c06c89199146e58d856c4d76166c64 in term 0.
I20260811 22:21:52.116513 21070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "408c91b893b64ddf8ed0da0bdfb90b41" candidate_uuid: "88c06c89199146e58d856c4d76166c64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "507253424ae14ca8a5845f55cad381f0"
I20260811 22:21:52.116809 21070 raft_consensus.cc:3037] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:52.115031 20650 raft_consensus.cc:2454] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88c06c89199146e58d856c4d76166c64 in term 1.
I20260811 22:21:52.118078 20768 leader_election.cc:302] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88c06c89199146e58d856c4d76166c64, d28213ca759a4e6fb8e4581fb2739826; no voters:
I20260811 22:21:52.118458 21197 raft_consensus.cc:2781] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:52.120712 21070 raft_consensus.cc:2454] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88c06c89199146e58d856c4d76166c64 in term 1.
I20260811 22:21:52.121433 21197 raft_consensus.cc:686] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [term 1 LEADER]: Becoming Leader. State: Replica: 88c06c89199146e58d856c4d76166c64, State: Running, Role: LEADER
I20260811 22:21:52.122589 21197 consensus_queue.cc:237] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d28213ca759a4e6fb8e4581fb2739826" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36731 } } peers { permanent_uuid: "88c06c89199146e58d856c4d76166c64" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 34245 } } peers { permanent_uuid: "507253424ae14ca8a5845f55cad381f0" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42487 } }
I20260811 22:21:52.132304 20404 catalog_manager.cc:5495] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88c06c89199146e58d856c4d76166c64 (127.29.229.2). New cstate: current_term: 1 leader_uuid: "88c06c89199146e58d856c4d76166c64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d28213ca759a4e6fb8e4581fb2739826" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88c06c89199146e58d856c4d76166c64" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 34245 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "507253424ae14ca8a5845f55cad381f0" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42487 } health_report { overall_health: UNKNOWN } } }
I20260811 22:21:52.159328 21181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:52.152616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40630 (local address 127.29.229.2:34245)
0811 22:21:52.152876 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:52.152883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:52.152906 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:21:52.153053 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:52.153058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:52.153185 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:52.153365 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:21:52.153743 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:52.154909 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:52.156346 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:52.157055 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:52.157195 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:52.157842 (+ 647us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:21:52.158590 (+ 748us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:21:52.158756 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:52.158925 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 22:21:52.159028 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 22:21:52.169342 21070 raft_consensus.cc:1261] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0 [term 1 FOLLOWER]: Refusing update from remote peer 88c06c89199146e58d856c4d76166c64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:52.169337 20650 raft_consensus.cc:1261] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826 [term 1 FOLLOWER]: Refusing update from remote peer 88c06c89199146e58d856c4d76166c64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:52.170711 21200 consensus_queue.cc:1035] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d28213ca759a4e6fb8e4581fb2739826" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:52.171187 21197 consensus_queue.cc:1035] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "507253424ae14ca8a5845f55cad381f0" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:52.211515 20404 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:52.211805 20404 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:21:52.217428 20404 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 634f51bfe49f4ee9a7d939843910b8ba: Sending DeleteTablet for 3 replicas of tablet 408c91b893b64ddf8ed0da0bdfb90b41
I20260811 22:21:52.218735 20613 tablet_service.cc:1514] Processing DeleteTablet for tablet 408c91b893b64ddf8ed0da0bdfb90b41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:52 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:57126
I20260811 22:21:52.218973 21046 tablet_service.cc:1514] Processing DeleteTablet for tablet 408c91b893b64ddf8ed0da0bdfb90b41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:52 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:51028
I20260811 22:21:52.220359 20839 tablet_service.cc:1514] Processing DeleteTablet for tablet 408c91b893b64ddf8ed0da0bdfb90b41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:21:52 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:40626
I20260811 22:21:52.223292 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 20458
I20260811 22:21:52.252740 21177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:52.226537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57136 (local address 127.29.229.1:36731)
0811 22:21:52.226705 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:52.226712 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:52.226731 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:21:52.226793 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:52.226797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:52.226892 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:52.226997 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:52.229016 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:52.230295 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:52.234276 (+ 3981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:52.235174 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:52.235324 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:52.242595 (+ 7271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:52.242617 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:52.242642 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:52.242701 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:52.247496 (+ 4795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:52.248721 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:52.248734 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:52.248740 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:52.248960 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:52.250678 (+ 1718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:52.250686 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:52.250692 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:52.251295 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:52.251642 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:52.252024 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 22:21:52.252277 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:21:52.253508 20707 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:21:52.260267 21231 tablet_replica.cc:330] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64: stopping tablet replica
I20260811 22:21:52.261145 21231 raft_consensus.cc:2229] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:21:52.262168 21231 raft_consensus.cc:2258] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:52.264761 21231 ts_tablet_manager.cc:1905] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:52.277316 21229 tablet_replica.cc:330] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0: stopping tablet replica
I20260811 22:21:52.278159 21229 raft_consensus.cc:2229] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:52.278975 21229 raft_consensus.cc:2258] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:52.281783 21229 ts_tablet_manager.cc:1905] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:52.279770 21231 ts_tablet_manager.cc:1918] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:52.286998 21231 log.cc:1196] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/408c91b893b64ddf8ed0da0bdfb90b41
I20260811 22:21:52.287823 21231 ts_tablet_manager.cc:1939] T 408c91b893b64ddf8ed0da0bdfb90b41 P 88c06c89199146e58d856c4d76166c64: Deleting consensus metadata
I20260811 22:21:52.290087 20379 catalog_manager.cc:4841] TS 88c06c89199146e58d856c4d76166c64 (127.29.229.2:34245): tablet 408c91b893b64ddf8ed0da0bdfb90b41 (table test-table [id=7ac24e2243b94c76b73ec18205461959]) successfully deleted
I20260811 22:21:52.294698 21229 ts_tablet_manager.cc:1918] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:52.295032 21229 log.cc:1196] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/408c91b893b64ddf8ed0da0bdfb90b41
I20260811 22:21:52.295723 21229 ts_tablet_manager.cc:1939] T 408c91b893b64ddf8ed0da0bdfb90b41 P 507253424ae14ca8a5845f55cad381f0: Deleting consensus metadata
I20260811 22:21:52.297956 20381 catalog_manager.cc:4841] TS 507253424ae14ca8a5845f55cad381f0 (127.29.229.3:42487): tablet 408c91b893b64ddf8ed0da0bdfb90b41 (table test-table [id=7ac24e2243b94c76b73ec18205461959]) successfully deleted
I20260811 22:21:52.436916 21227 tablet_replica.cc:330] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826: stopping tablet replica
I20260811 22:21:52.437407 21227 raft_consensus.cc:2229] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:21:52.437955 21227 raft_consensus.cc:2258] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:21:52.439985 21227 ts_tablet_manager.cc:1905] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:21:52.447414 21227 ts_tablet_manager.cc:1918] T 408c91b893b64ddf8ed0da0bdfb90b41 P d28213ca759a4e6fb8e4581fb2739826: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:21:52.448160 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 20458
W20260811 22:21:52.483700 20381 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.229.1:36731 (error 108)
I20260811 22:21:52.484644 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 20722
I20260811 22:21:52.504958 21181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:52.486143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40646 (local address 127.29.229.2:34245)
0811 22:21:52.486308 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:52.486313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:52.486331 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:21:52.486395 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:52.486400 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:52.486499 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:52.486620 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:52.488453 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:52.489636 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:52.491533 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:52.492177 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:52.492299 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:52.496063 (+ 3764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:52.496081 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:52.496102 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:52.496150 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:52.500667 (+ 4517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:52.501574 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:52.501583 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:52.501587 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:52.501777 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:52.502259 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:52.502263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:52.502266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:52.502747 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:52.503069 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:52.504290 (+ 1221us) server_negotiation.cc:299] Negotiation successful
0811 22:21:52.504562 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:21:52.506026 20922 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:21:52.506870 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:52.500012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:36731 (local address 127.0.0.1:57152)
0811 22:21:52.506521 (+ 6509us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:52.506669 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:36731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6350,"threads_started":1}
W20260811 22:21:52.507225 20381 catalog_manager.cc:4584] TS d28213ca759a4e6fb8e4581fb2739826 (127.29.229.1:36731): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 408c91b893b64ddf8ed0da0bdfb90b41: Network error: Client connection negotiation failed: client connection to 127.29.229.1:36731: connect: Connection refused (error 111)
I20260811 22:21:52.561393 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:52.560771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:36731 (local address 127.0.0.1:57166)
0811 22:21:52.561076 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:52.561219 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:36731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
I20260811 22:21:52.562614 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:52.562205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:36731 (local address 127.0.0.1:57178)
0811 22:21:52.562356 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:52.562444 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:36731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 22:21:52.623247 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:52.622775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:36731 (local address 127.0.0.1:57188)
0811 22:21:52.622996 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:52.623092 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:36731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 22:21:52.624325 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:52.623984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:36731 (local address 127.0.0.1:57190)
0811 22:21:52.624122 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:52.624192 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:36731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 22:21:52.639623 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 20722
I20260811 22:21:52.669922 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 20937
I20260811 22:21:52.684377 21236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:52.670664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51042 (local address 127.29.229.3:42487)
0811 22:21:52.671180 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:52.671184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:52.671201 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:21:52.671257 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:52.671261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:52.671334 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:52.671426 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:52.672875 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:52.673669 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:52.675066 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:52.675441 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:52.675537 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:52.678774 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:52.678786 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:52.678802 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:52.678839 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:52.681685 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:52.682499 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:52.682507 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:52.682510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:52.682650 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:52.683007 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:52.683012 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:52.683014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:52.683402 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:52.683631 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:52.683891 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 22:21:52.684072 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":307,"threads_started":1}
I20260811 22:21:52.685091 21138 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:21:52.704938 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:52.704407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:36731 (local address 127.0.0.1:57194)
0811 22:21:52.704678 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:52.704781 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:36731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260811 22:21:52.706053 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:52.705672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:36731 (local address 127.0.0.1:57198)
0811 22:21:52.705861 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:52.705928 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:36731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 22:21:52.776688 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 20937
I20260811 22:21:52.802786 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 20351
I20260811 22:21:52.803982 20449 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:21:52.924321 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:52.923767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:36731 (local address 127.0.0.1:57212)
0811 22:21:52.923982 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:52.924116 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:36731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 22:21:52.924394 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 20351
I20260811 22:21:52.957722 30612 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 22:21:53.315394 30612 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 22:21:53.315615 30612 mini_ranger.cc:66] Stopping Ranger...
I20260811 22:21:53.672672 30612 mini_ranger.cc:68] Stopped Ranger
2026-08-11 22:21:53.673 UTC [18825] LOG: received smart shutdown request
2026-08-11 22:21:53.676 UTC [18825] LOG: background worker "logical replication launcher" (PID 18836) exited with exit code 1
2026-08-11 22:21:53.676 UTC [18830] LOG: shutting down
2026-08-11 22:21:53.855 UTC [18825] LOG: database system is shut down
2026-08-11T22:21:53Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (59544 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:21:54 dist-test-slave-jq4t krb5kdc[21258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:21:54 dist-test-slave-jq4t krb5kdc[21258](info): set up 2 sockets
Aug 11 22:21:54 dist-test-slave-jq4t krb5kdc[21258](info): commencing operation
krb5kdc: starting...
W20260811 22:21:55.388513 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.356s user 0.003s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:21:55 dist-test-slave-jq4t krb5kdc[21258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:21:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:21:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:21:55.737845 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:42551
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:41253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:42551,127.29.229.61:45707,127.29.229.60:46053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:21:56.117285 21281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:56.117627 21281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:56.117836 21281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:56.126717 21281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:56.126879 21281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:56.126947 21281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:56.127095 21281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:56.127174 21281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:56.137894 21281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:41253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:42551,127.29.229.61:45707,127.29.229.60:46053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:42551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:21:56.139791 21281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:56.142256 21281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:56.156075 21295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:56.156230 21292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:56.156294 21291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:56.157490 21281 server_base.cc:1034] running on GCE node
I20260811 22:21:56.158388 21281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:56.159972 21281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:56.161209 21281 hybrid_clock.cc:648] HybridClock initialized: now 1786486916161169 us; error 64 us; skew 500 ppm
I20260811 22:21:56.161643 21281 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:21:56 dist-test-slave-jq4t krb5kdc[21258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486916, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:56.168032 21281 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:21:56.169788 21281 webserver.cc:460] Webserver started at http://127.29.229.62:33269/ using document root <none> and password file <none>
I20260811 22:21:56.170507 21281 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:56.170617 21281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:56.170902 21281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:56.174226 21281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "09d6e7fa36d54a60bea43493a59535cb"
format_stamp: "Formatted at 2026-08-11 22:21:56 on dist-test-slave-jq4t"
server_key: "88550dc4a7a1d63338b54895e421b8c7"
server_key_iv: "7b4441c851dd77c1aeb0c30d8bccf5a5"
server_key_version: "encryptionkey@0"
I20260811 22:21:56.175057 21281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "09d6e7fa36d54a60bea43493a59535cb"
format_stamp: "Formatted at 2026-08-11 22:21:56 on dist-test-slave-jq4t"
server_key: "88550dc4a7a1d63338b54895e421b8c7"
server_key_iv: "7b4441c851dd77c1aeb0c30d8bccf5a5"
server_key_version: "encryptionkey@0"
I20260811 22:21:56.181344 21281 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 22:21:56.185644 21311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:56.187202 21281 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:21:56.187378 21281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "09d6e7fa36d54a60bea43493a59535cb"
format_stamp: "Formatted at 2026-08-11 22:21:56 on dist-test-slave-jq4t"
server_key: "88550dc4a7a1d63338b54895e421b8c7"
server_key_iv: "7b4441c851dd77c1aeb0c30d8bccf5a5"
server_key_version: "encryptionkey@0"
I20260811 22:21:56.187577 21281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:56.217710 21281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:56.223235 21281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:56.223614 21281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:56.247972 21281 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:42551
I20260811 22:21:56.248015 21389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:42551 every 8 connection(s)
I20260811 22:21:56.249511 21281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:21:56.254158 21391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:56.254585 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 21281
I20260811 22:21:56.254943 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:21:56.256181 30612 external_mini_cluster.cc:1428] Setting key a27f27ee8d8bfc19129f62bfce0b92ed
I20260811 22:21:56.261559 21391 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:21:56 dist-test-slave-jq4t krb5kdc[21258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486916, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:21:56.287112 21394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.265031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39332 (local address 127.29.229.62:42551)
0811 22:21:56.265996 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:56.266016 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:56.266474 (+ 458us) server_negotiation.cc:407] Connection header received
0811 22:21:56.268672 (+ 2198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:56.268678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:56.268915 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:56.269134 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:56.270522 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.271796 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.273315 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.273902 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.274129 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:56.278812 (+ 4683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:56.278820 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:56.278833 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:56.278865 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:56.281871 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:56.282929 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:56.282937 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:56.282939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:56.283110 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:56.283690 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:56.283697 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:56.283700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:56.284312 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:56.284556 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:56.285621 (+ 1065us) server_negotiation.cc:299] Negotiation successful
0811 22:21:56.286076 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":111,"server-negotiator.queue_time_us":690,"thread_start_us":447,"threads_started":1}
I20260811 22:21:56.287156 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.265043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:42551 (local address 127.0.0.1:39332)
0811 22:21:56.266293 (+ 1250us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.266328 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:56.268247 (+ 1919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:56.269158 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:56.269164 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:56.269950 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:56.270302 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:56.270319 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.271977 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.271982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:56.273168 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:56.273181 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.274248 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.274260 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:56.274491 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:56.275254 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:56.275290 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:56.278540 (+ 3250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:56.282060 (+ 3520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:56.282077 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:56.282093 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:56.282742 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:56.283255 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:56.283263 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:56.283270 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:56.283513 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:56.284518 (+ 1005us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:56.284531 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:56.284852 (+ 321us) client_negotiation.cc:769] Sending connection context
0811 22:21:56.285209 (+ 357us) client_negotiation.cc:240] Negotiation successful
0811 22:21:56.285886 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1043,"thread_start_us":816,"threads_started":1}
WARNING: no policy specified for kudu/127.29.229.61@KRBTEST.COM; defaulting to no policy
I20260811 22:21:56.290483 21391 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.61@KRBTEST.COM" created.
I20260811 22:21:56.292366 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.291749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45707 (local address 127.0.0.1:60770)
0811 22:21:56.292028 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.292159 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
W20260811 22:21:56.292809 21320 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45707: connect: Connection refused (error 111)
I20260811 22:21:56.294770 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.294321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45707 (local address 127.0.0.1:60778)
0811 22:21:56.294533 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.294624 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260811 22:21:56.295228 21391 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.61:45707: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45707: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.61.keytab.
Entry for principal kudu/127.29.229.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.61.keytab.
I20260811 22:21:56.332926 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.61:45707
--webserver_interface=127.29.229.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.61.keytab
--principal=kudu/127.29.229.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:41253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:42551,127.29.229.61:45707,127.29.229.60:46053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:21:56.344892 21391 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } attempt: 1
I20260811 22:21:56.346655 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.346038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45707 (local address 127.0.0.1:60782)
0811 22:21:56.346306 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.346468 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 22:21:56.347922 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.347488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45707 (local address 127.0.0.1:60792)
0811 22:21:56.347658 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.347749 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 22:21:56.348166 21391 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.61:45707: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45707: connect: Connection refused (error 111)
I20260811 22:21:56.416368 21391 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } attempt: 2
I20260811 22:21:56.417920 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.417262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45707 (local address 127.0.0.1:60794)
0811 22:21:56.417558 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.417689 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 22:21:56.421387 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.418777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45707 (local address 127.0.0.1:60798)
0811 22:21:56.421015 (+ 2238us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.421160 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2074}
W20260811 22:21:56.421765 21391 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.61:45707: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45707: connect: Connection refused (error 111)
I20260811 22:21:56.513123 21391 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } attempt: 3
I20260811 22:21:56.514632 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.514141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45707 (local address 127.0.0.1:60812)
0811 22:21:56.514399 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.514497 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260811 22:21:56.515733 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.515352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45707 (local address 127.0.0.1:60818)
0811 22:21:56.515516 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.515596 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 22:21:56.516002 21391 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.61:45707: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45707: connect: Connection refused (error 111)
I20260811 22:21:56.659328 21391 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } attempt: 4
I20260811 22:21:56.660742 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.660082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45707 (local address 127.0.0.1:60832)
0811 22:21:56.660429 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.660591 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
I20260811 22:21:56.661842 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.661461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45707 (local address 127.0.0.1:60846)
0811 22:21:56.661610 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.661720 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 22:21:56.662102 21391 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.61:45707: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45707: connect: Connection refused (error 111)
W20260811 22:21:56.753300 21400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:56.753733 21400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:56.753953 21400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:56.761514 21400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:56.761641 21400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:56.761746 21400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:56.761852 21400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:56.761942 21400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:56.772399 21400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:41253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:42551,127.29.229.61:45707,127.29.229.60:46053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.61.keytab
--principal=kudu/127.29.229.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.61:45707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.229.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:21:56.773922 21400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:56.775328 21400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:56.789768 21410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:56.789681 21413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:56.789719 21416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:56.790647 21400 server_base.cc:1034] running on GCE node
I20260811 22:21:56.791580 21400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:56.793143 21400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:56.794414 21400 hybrid_clock.cc:648] HybridClock initialized: now 1786486916794350 us; error 62 us; skew 500 ppm
I20260811 22:21:56.794807 21400 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:21:56 dist-test-slave-jq4t krb5kdc[21258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486916, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:56.799746 21400 init.cc:377] Logged in from keytab as kudu/127.29.229.61@KRBTEST.COM (short username kudu)
I20260811 22:21:56.801525 21400 webserver.cc:460] Webserver started at http://127.29.229.61:40077/ using document root <none> and password file <none>
I20260811 22:21:56.802280 21400 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:56.802436 21400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:56.802773 21400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:56.806046 21400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-1/data/instance:
uuid: "efc6cc15b503419e9d54b3974204cfd5"
format_stamp: "Formatted at 2026-08-11 22:21:56 on dist-test-slave-jq4t"
server_key: "86dabd15b1a38220ef482d698ab88cfb"
server_key_iv: "0bbee138af913ff6ced817a2325af2b4"
server_key_version: "encryptionkey@0"
I20260811 22:21:56.806891 21400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-1/wal/instance:
uuid: "efc6cc15b503419e9d54b3974204cfd5"
format_stamp: "Formatted at 2026-08-11 22:21:56 on dist-test-slave-jq4t"
server_key: "86dabd15b1a38220ef482d698ab88cfb"
server_key_iv: "0bbee138af913ff6ced817a2325af2b4"
server_key_version: "encryptionkey@0"
I20260811 22:21:56.812672 21400 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 22:21:56.816653 21426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:56.817992 21400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 22:21:56.818184 21400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-1/wal
uuid: "efc6cc15b503419e9d54b3974204cfd5"
format_stamp: "Formatted at 2026-08-11 22:21:56 on dist-test-slave-jq4t"
server_key: "86dabd15b1a38220ef482d698ab88cfb"
server_key_iv: "0bbee138af913ff6ced817a2325af2b4"
server_key_version: "encryptionkey@0"
I20260811 22:21:56.818397 21400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:56.838683 21400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:56.844439 21400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:56.844877 21400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:56.866621 21400 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.61:45707
I20260811 22:21:56.866717 21511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.61:45707 every 8 connection(s)
I20260811 22:21:56.868204 21400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-1/data/info.pb
I20260811 22:21:56.872373 21512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:56.876435 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 21400
I20260811 22:21:56.876719 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-1/wal/instance
I20260811 22:21:56.877230 30612 external_mini_cluster.cc:1428] Setting key acf0973f9b89a80ac5620743a092a6d1
I20260811 22:21:56.879590 21512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:21:56 dist-test-slave-jq4t krb5kdc[21258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486916, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.61@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:21:56.904723 21514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.882674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39338 (local address 127.29.229.62:42551)
0811 22:21:56.883241 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:56.883250 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:56.884291 (+ 1041us) server_negotiation.cc:407] Connection header received
0811 22:21:56.886291 (+ 2000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:56.886299 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:56.886408 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:56.886538 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:56.888186 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.889400 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.891061 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.891586 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.891697 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:56.897050 (+ 5353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:56.897059 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:56.897063 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:56.897143 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:56.900434 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:56.901674 (+ 1240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:56.901679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:56.901682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:56.901831 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:56.902511 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:56.902515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:56.902518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:56.902840 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:56.902996 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:56.904012 (+ 1016us) server_negotiation.cc:299] Negotiation successful
0811 22:21:56.904347 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":269,"threads_started":1}
I20260811 22:21:56.906194 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.883264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:42551 (local address 127.0.0.1:39338)
0811 22:21:56.884104 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.884155 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:56.885999 (+ 1844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:56.886845 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:56.886854 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:56.887399 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:56.887970 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:56.887988 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.889585 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.889589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:56.890914 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:56.890928 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.891844 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.891852 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:56.892240 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:56.893113 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:56.893149 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:56.896569 (+ 3420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:56.900698 (+ 4129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:56.900711 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:56.900735 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:56.901501 (+ 766us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:56.901986 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:56.901994 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:56.902001 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:56.902322 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:56.902994 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:56.903006 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:56.903550 (+ 544us) client_negotiation.cc:769] Sending connection context
0811 22:21:56.903911 (+ 361us) client_negotiation.cc:240] Negotiation successful
0811 22:21:56.904603 (+ 692us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":359,"threads_started":1}
I20260811 22:21:56.908249 21512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.229.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.60@KRBTEST.COM" created.
Aug 11 22:21:56 dist-test-slave-jq4t krb5kdc[21258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486916, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.61@KRBTEST.COM for kudu/127.29.229.61@KRBTEST.COM
I20260811 22:21:56.928409 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.909268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45707 (local address 127.0.0.1:60858)
0811 22:21:56.909487 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.909508 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:56.909631 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:56.910477 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:56.910482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:56.911333 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:56.911571 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:56.911590 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.913858 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.913864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:56.915068 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:56.915084 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.916724 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.916730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:56.917010 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:56.917794 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:56.917814 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:56.919986 (+ 2172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:56.925017 (+ 5031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:56.925023 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:56.925028 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:56.925482 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:56.925917 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:56.925928 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:56.925931 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:56.926091 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:56.927307 (+ 1216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:56.927315 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:56.927517 (+ 202us) client_negotiation.cc:769] Sending connection context
0811 22:21:56.927677 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 22:21:56.928002 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 22:21:56.928730 21520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.909344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60858 (local address 127.29.229.61:45707)
0811 22:21:56.910018 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:56.910025 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:56.910051 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:21:56.910099 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:56.910105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:56.910287 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:56.910409 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:56.911923 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.913628 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.915331 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.916517 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.916682 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:56.920334 (+ 3652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:56.920359 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:56.920389 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:56.920457 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:56.924841 (+ 4384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:56.925629 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:56.925642 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:56.925644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:56.925768 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:56.926214 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:56.926221 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:56.926224 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:56.927025 (+ 801us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:56.927429 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:56.927839 (+ 410us) server_negotiation.cc:299] Negotiation successful
0811 22:21:56.928155 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":501,"thread_start_us":342,"threads_started":1}
I20260811 22:21:56.931643 21512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:21:56.933178 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.932471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41650)
0811 22:21:56.932752 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.932935 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 22:21:56.933539 21427 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
I20260811 22:21:56.935309 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.934907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41666)
0811 22:21:56.935099 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.935188 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260811 22:21:56.935702 21512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:46053: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.60.keytab.
Entry for principal kudu/127.29.229.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.60.keytab.
I20260811 22:21:56.961406 21391 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } attempt: 5
I20260811 22:21:56.961714 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.60:46053
--webserver_interface=127.29.229.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.60.keytab
--principal=kudu/127.29.229.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:41253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:42551,127.29.229.61:45707,127.29.229.60:46053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:21:56 dist-test-slave-jq4t krb5kdc[21258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486916, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for kudu/127.29.229.61@KRBTEST.COM
I20260811 22:21:56.977283 21520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.962447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60862 (local address 127.29.229.61:45707)
0811 22:21:56.962677 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:56.962682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:56.962704 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:21:56.962917 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:56.962921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:56.962988 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:56.963099 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:56.963909 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.964672 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.965857 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.966315 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.966458 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:56.969745 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:56.969755 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:56.969764 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:56.969816 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:56.974375 (+ 4559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:56.975183 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:56.975191 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:56.975197 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:56.975355 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:56.975731 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:56.975738 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:56.975743 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:56.976165 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:56.976334 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:56.976878 (+ 544us) server_negotiation.cc:299] Negotiation successful
0811 22:21:56.977054 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 22:21:56.977582 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.962316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45707 (local address 127.0.0.1:60862)
0811 22:21:56.962575 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.962602 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:56.962737 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:56.963134 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:56.963138 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:56.963562 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:56.963734 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:56.963743 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.964808 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.964814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:56.965701 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:56.965713 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:56.966475 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:56.966482 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:56.966662 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:56.967344 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:56.967362 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:56.969513 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:56.974593 (+ 5080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:56.974598 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:56.974601 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:56.975044 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:56.975479 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:56.975482 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:56.975484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:56.975624 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:56.976324 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:56.976332 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:56.976565 (+ 233us) client_negotiation.cc:769] Sending connection context
0811 22:21:56.976772 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 22:21:56.977157 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260811 22:21:56.978639 21391 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:21:56.979799 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.979294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41682)
0811 22:21:56.979506 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.979613 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 22:21:56.981361 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.980859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41688)
0811 22:21:56.981094 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.981205 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 22:21:56.981618 21391 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:46053: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
I20260811 22:21:56.985069 21512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } attempt: 1
I20260811 22:21:56.986248 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.985765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41690)
0811 22:21:56.985994 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.986099 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260811 22:21:56.987397 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:56.987030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41702)
0811 22:21:56.987207 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:56.987278 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 22:21:56.987645 21512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:46053: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
I20260811 22:21:57.032891 21391 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } attempt: 1
I20260811 22:21:57.034920 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.033799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41716)
0811 22:21:57.034336 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.034509 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
I20260811 22:21:57.036327 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.035842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41720)
0811 22:21:57.036044 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.036173 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 22:21:57.036659 21391 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:46053: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
I20260811 22:21:57.055985 21512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } attempt: 2
I20260811 22:21:57.057531 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.057025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41726)
0811 22:21:57.057253 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.057360 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 22:21:57.058635 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.058283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41734)
0811 22:21:57.058437 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.058517 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 22:21:57.058853 21512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:46053: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
I20260811 22:21:57.104918 21391 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } attempt: 2
I20260811 22:21:57.106324 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.105663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41746)
0811 22:21:57.105979 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.106104 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 22:21:57.107537 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.107153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41758)
0811 22:21:57.107315 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.107403 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 22:21:57.107817 21391 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:46053: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
I20260811 22:21:57.150138 21512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } attempt: 3
I20260811 22:21:57.151460 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.150930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41768)
0811 22:21:57.151204 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.151295 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260811 22:21:57.153076 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.152294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41782)
0811 22:21:57.152751 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.152891 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287}
W20260811 22:21:57.153394 21512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:46053: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
I20260811 22:21:57.214107 21391 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } attempt: 3
I20260811 22:21:57.215421 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.214844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41796)
0811 22:21:57.215094 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.215208 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 22:21:57.216923 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.216388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41802)
0811 22:21:57.216709 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.216790 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184}
W20260811 22:21:57.217206 21391 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:46053: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
I20260811 22:21:57.296651 21512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } attempt: 4
I20260811 22:21:57.297972 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.297434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41816)
0811 22:21:57.297695 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.297793 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 22:21:57.298944 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.298625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41828)
0811 22:21:57.298777 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.298845 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:21:57.299234 21512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:46053: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
W20260811 22:21:57.342895 21523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:21:57.343199 21523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:21:57.343360 21523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:21:57.350886 21523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:21:57.351009 21523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:21:57.351091 21523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:21:57.351176 21523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:21:57.351274 21523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:21:57.361608 21523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:41253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:42551,127.29.229.61:45707,127.29.229.60:46053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.60.keytab
--principal=kudu/127.29.229.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.60:46053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.229.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:21:57.363003 21523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:21:57.364349 21523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:21:57.376852 21528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:57.376850 21532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:21:57.376987 21529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:21:57.378213 21523 server_base.cc:1034] running on GCE node
I20260811 22:21:57.379159 21523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:21:57.381403 21523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:21:57.382731 21523 hybrid_clock.cc:648] HybridClock initialized: now 1786486917382625 us; error 58 us; skew 500 ppm
I20260811 22:21:57.383526 21523 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:21:57 dist-test-slave-jq4t krb5kdc[21258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486917, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:21:57.388265 21523 init.cc:377] Logged in from keytab as kudu/127.29.229.60@KRBTEST.COM (short username kudu)
I20260811 22:21:57.391113 21523 webserver.cc:460] Webserver started at http://127.29.229.60:33125/ using document root <none> and password file <none>
I20260811 22:21:57.391999 21523 fs_manager.cc:359] Metadata directory not provided
I20260811 22:21:57.392172 21523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:21:57.392603 21523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:21:57.394454 21391 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } attempt: 4
I20260811 22:21:57.395612 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.395110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41844)
0811 22:21:57.395356 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.395450 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 22:21:57.396225 21523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-2/data/instance:
uuid: "65da11e950c64f0283ee222ef8073fea"
format_stamp: "Formatted at 2026-08-11 22:21:57 on dist-test-slave-jq4t"
server_key: "bd9e401f5ff6e0ae8b23b73b3dfb5b99"
server_key_iv: "ad5c6aa1b2647c3663cfd478f05de5ee"
server_key_version: "encryptionkey@0"
I20260811 22:21:57.396713 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.396281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41858)
0811 22:21:57.396440 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.396572 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:21:57.396960 21391 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:46053: Network error: Client connection negotiation failed: client connection to 127.29.229.60:46053: connect: Connection refused (error 111)
I20260811 22:21:57.397300 21523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-2/wal/instance:
uuid: "65da11e950c64f0283ee222ef8073fea"
format_stamp: "Formatted at 2026-08-11 22:21:57 on dist-test-slave-jq4t"
server_key: "bd9e401f5ff6e0ae8b23b73b3dfb5b99"
server_key_iv: "ad5c6aa1b2647c3663cfd478f05de5ee"
server_key_version: "encryptionkey@0"
I20260811 22:21:57.403151 21523 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:21:57.407392 21541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:21:57.409284 21523 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:21:57.409507 21523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-2/wal
uuid: "65da11e950c64f0283ee222ef8073fea"
format_stamp: "Formatted at 2026-08-11 22:21:57 on dist-test-slave-jq4t"
server_key: "bd9e401f5ff6e0ae8b23b73b3dfb5b99"
server_key_iv: "ad5c6aa1b2647c3663cfd478f05de5ee"
server_key_version: "encryptionkey@0"
I20260811 22:21:57.409775 21523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:21:57.421669 21523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:21:57.426578 21523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:21:57.427045 21523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:21:57.447368 21523 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.60:46053
I20260811 22:21:57.447460 21617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.60:46053 every 8 connection(s)
I20260811 22:21:57.448872 21523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-2/data/info.pb
I20260811 22:21:57.451592 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 21523
I20260811 22:21:57.451926 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486685119861-30612-0/minicluster-data/master-2/wal/instance
I20260811 22:21:57.452653 30612 external_mini_cluster.cc:1428] Setting key 97b46a3575dcca84a1099d1117d171b3
I20260811 22:21:57.454137 21618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:21:57.465237 21618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:21:57 dist-test-slave-jq4t krb5kdc[21258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.60@KRBTEST.COM
Aug 11 22:21:57 dist-test-slave-jq4t krb5kdc[21258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486917, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.60@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:21:57.480280 21623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.454926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41860 (local address 127.29.229.60:46053)
0811 22:21:57.455841 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:57.455859 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:57.455921 (+ 62us) server_negotiation.cc:407] Connection header received
0811 22:21:57.457889 (+ 1968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:57.457919 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:57.458501 (+ 582us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:57.458964 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:57.460391 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.462908 (+ 2517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.464351 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.465453 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.465664 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.471132 (+ 5468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:57.471140 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:57.471151 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:57.471187 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:57.475245 (+ 4058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.476322 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.476329 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.476335 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.476624 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.477127 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.477138 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.477140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.477859 (+ 719us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:57.478175 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:57.478695 (+ 520us) server_negotiation.cc:299] Negotiation successful
0811 22:21:57.479132 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":397,"threads_started":1}
I20260811 22:21:57.486096 21627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.468311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:42551 (local address 127.0.0.1:39340)
0811 22:21:57.469213 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.469254 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:57.469445 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:57.469896 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:57.469902 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:57.470518 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:57.470814 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:57.470913 (+ 99us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.472178 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.472189 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:57.474076 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:57.474092 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.474966 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.474975 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:57.475334 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.476050 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:57.476077 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:57.478890 (+ 2813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:57.482641 (+ 3751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:57.482674 (+ 33us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:57.482683 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:57.483302 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:57.483743 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:57.483751 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:57.483756 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:57.483936 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:57.484756 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:57.484765 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:57.485015 (+ 250us) client_negotiation.cc:769] Sending connection context
0811 22:21:57.485242 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 22:21:57.485626 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":555,"threads_started":1}
I20260811 22:21:57.486121 21624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.468258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39340 (local address 127.29.229.62:42551)
0811 22:21:57.468925 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:57.468934 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:57.469354 (+ 420us) server_negotiation.cc:407] Connection header received
0811 22:21:57.469579 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:57.469585 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:57.469671 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:57.469790 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:57.471077 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.471985 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.474237 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.474651 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.474841 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.479100 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:57.479105 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:57.479109 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:57.479144 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:57.482480 (+ 3336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.483453 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.483463 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.483465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.483606 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.484129 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.484136 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.484140 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.484590 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:57.484765 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:57.485392 (+ 627us) server_negotiation.cc:299] Negotiation successful
0811 22:21:57.485562 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":339,"threads_started":1}
I20260811 22:21:57.487835 21618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:21:57 dist-test-slave-jq4t krb5kdc[21258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
Aug 11 22:21:57 dist-test-slave-jq4t krb5kdc[21258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486917, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.60@KRBTEST.COM for kudu/127.29.229.61@KRBTEST.COM
I20260811 22:21:57.501806 21628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.483801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39344 (local address 127.29.229.62:42551)
0811 22:21:57.484503 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:57.484509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:57.484533 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:21:57.484581 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:57.484585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:57.484659 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:57.484766 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:57.486900 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.488151 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.490013 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.490598 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.490694 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.495971 (+ 5277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:57.495976 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:57.495979 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:57.496013 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:57.498768 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.499590 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.499594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.499596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.499718 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.500154 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.500157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.500159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.500558 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:57.500828 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:57.501074 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 22:21:57.501351 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":312,"threads_started":1}
I20260811 22:21:57.503434 21627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.488636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45707 (local address 127.0.0.1:60864)
0811 22:21:57.488826 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.488847 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:57.488979 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:57.489614 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:57.489618 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:57.490045 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:57.490223 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:57.490233 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.491641 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.491646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:57.492804 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:57.492818 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.493876 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.493881 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:57.494069 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.494649 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:57.494672 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:57.496764 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:57.500415 (+ 3651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:57.500421 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:57.500428 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:57.501001 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:57.501491 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:57.501496 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:57.501502 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:57.501677 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:57.502375 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:57.502380 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:57.502624 (+ 244us) client_negotiation.cc:769] Sending connection context
0811 22:21:57.502809 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 22:21:57.503098 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 22:21:57.503530 21629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.488752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60864 (local address 127.29.229.61:45707)
0811 22:21:57.489280 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:57.489287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:57.489306 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:21:57.489363 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:57.489369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:57.489446 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:57.489576 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:57.490386 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.491456 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.492960 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.493690 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.493828 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.496960 (+ 3132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:57.496965 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:57.496969 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:57.497002 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:57.500276 (+ 3274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.501169 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.501177 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.501182 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.501360 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.501816 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.501824 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.501827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.502232 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:57.502403 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:57.502967 (+ 564us) server_negotiation.cc:299] Negotiation successful
0811 22:21:57.503154 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":269,"threads_started":1}
I20260811 22:21:57.505511 21618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:21:57 dist-test-slave-jq4t krb5kdc[21258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486917, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.60@KRBTEST.COM for kudu/127.29.229.60@KRBTEST.COM
Aug 11 22:21:57 dist-test-slave-jq4t krb5kdc[21258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.61@KRBTEST.COM
I20260811 22:21:57.520367 21627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.506418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41868)
0811 22:21:57.506593 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.506612 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:57.506715 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:57.507050 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:57.507053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:57.507403 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:57.507533 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:57.507541 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.509261 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.509268 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:57.510223 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:57.510240 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.511114 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.511120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:57.511336 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.511875 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:57.511891 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:57.514126 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:57.517490 (+ 3364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:57.517496 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:57.517500 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:57.518008 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:57.518392 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:57.518398 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:57.518411 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:57.518581 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:57.519313 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:57.519317 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:57.519498 (+ 181us) client_negotiation.cc:769] Sending connection context
0811 22:21:57.519681 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 22:21:57.519991 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 22:21:57.520663 21623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.506510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41868 (local address 127.29.229.60:46053)
0811 22:21:57.506670 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:57.506677 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:57.506694 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:21:57.506809 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:57.506814 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:57.506904 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:57.507005 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:57.507672 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.509020 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.510396 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.510835 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.511095 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.514325 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:57.514332 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:57.514337 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:57.514383 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:57.517340 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.518145 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.518150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.518152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.518276 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.518755 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.518762 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.518765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.519174 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:57.519360 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:57.519801 (+ 441us) server_negotiation.cc:299] Negotiation successful
0811 22:21:57.520228 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":111,"server-negotiator.queue_time_us":58}
I20260811 22:21:57.522017 21629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.504764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60874 (local address 127.29.229.61:45707)
0811 22:21:57.505096 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:57.505101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:57.505119 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:21:57.505173 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:57.505181 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:57.505264 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:57.505383 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:57.506898 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.507749 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.509304 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.509972 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.510099 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.515363 (+ 5264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:57.515372 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:57.515377 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:57.515426 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:57.518612 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.519515 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.519523 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.519527 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.519687 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.520172 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.520177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.520181 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.520611 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:57.520921 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:57.521426 (+ 505us) server_negotiation.cc:299] Negotiation successful
0811 22:21:57.521661 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":421,"threads_started":1}
I20260811 22:21:57.527177 21618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea: Bootstrap starting.
I20260811 22:21:57.527443 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:21:57.530884 21618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:57.532052 21618 log.cc:824] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:57.537601 21618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea: No bootstrap required, opened a new log
I20260811 22:21:57.550207 21618 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } } peers { permanent_uuid: "efc6cc15b503419e9d54b3974204cfd5" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } } peers { permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } }
I20260811 22:21:57.550724 21618 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:57.550884 21618 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65da11e950c64f0283ee222ef8073fea, State: Initialized, Role: FOLLOWER
I20260811 22:21:57.551640 21618 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } } peers { permanent_uuid: "efc6cc15b503419e9d54b3974204cfd5" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } } peers { permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } }
I20260811 22:21:57.555294 21623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.534143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41874 (local address 127.29.229.60:46053)
0811 22:21:57.534355 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:57.534362 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:57.535601 (+ 1239us) server_negotiation.cc:407] Connection header received
0811 22:21:57.535939 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:57.535944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:57.536020 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:57.536089 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:57.538598 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.539678 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.541854 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.542617 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.542724 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.547030 (+ 4306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:57.547037 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:57.547042 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:57.547090 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:57.551412 (+ 4322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.552418 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.552424 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.552428 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.552657 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.553337 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.553343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.553348 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.553765 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:57.554029 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:57.554656 (+ 627us) server_negotiation.cc:299] Negotiation successful
0811 22:21:57.554906 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:21:57.555374 21629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.534602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60876 (local address 127.29.229.61:45707)
0811 22:21:57.534762 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:57.534767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:57.534786 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:21:57.534855 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:57.534861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:57.534964 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:57.535080 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:57.537694 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.538851 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.541255 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.541932 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.542036 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.548156 (+ 6120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:57.548166 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:57.548172 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:57.548222 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:57.551912 (+ 3690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.552968 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.552975 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.552980 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.553148 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.553638 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.553643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.553647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.554032 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:57.554261 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:57.554811 (+ 550us) server_negotiation.cc:299] Negotiation successful
0811 22:21:57.555025 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 22:21:57.558133 21628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.533494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39356 (local address 127.29.229.62:42551)
0811 22:21:57.533761 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:57.533771 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:57.534681 (+ 910us) server_negotiation.cc:407] Connection header received
0811 22:21:57.534870 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:57.534875 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:57.534947 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:57.535054 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:57.537693 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.538818 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.541255 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.541932 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.542036 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.547250 (+ 5214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:57.547258 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:57.547262 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:57.547310 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:57.551799 (+ 4489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.553265 (+ 1466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.553274 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.553280 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.553484 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.554254 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.554263 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.554268 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.554779 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:57.555093 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:57.555947 (+ 854us) server_negotiation.cc:299] Negotiation successful
0811 22:21:57.557648 (+ 1701us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:21:57.559706 21643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } } peers { permanent_uuid: "efc6cc15b503419e9d54b3974204cfd5" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } } peers { permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } } }
I20260811 22:21:57.560160 21643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:57.561131 21618 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:21:57.573017 21663 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:21:57.573184 21663 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:21:57.569468 21523 master_runner.cc:186] Error getting master registration for 127.29.229.62:42551: OK, instance_id { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" instance_seqno: 1786486916241368 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:21:57.580432 21523 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:21:57.598515 21512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } attempt: 5
Aug 11 22:21:57 dist-test-slave-jq4t krb5kdc[21258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486916, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.61@KRBTEST.COM for kudu/127.29.229.60@KRBTEST.COM
I20260811 22:21:57.615435 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.599403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41878)
0811 22:21:57.599634 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.599657 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:57.599784 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:57.600279 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:57.600284 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:57.600888 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:57.601126 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:57.601142 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.602697 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.602703 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:57.604111 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:57.604129 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.604933 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.604939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:57.605181 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.605898 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:57.605923 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:57.608713 (+ 2790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:57.612378 (+ 3665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:57.612385 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:57.612389 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:57.613000 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:57.613562 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:57.613568 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:57.613571 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:57.613745 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:57.614518 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:57.614523 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:57.614702 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 22:21:57.614881 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 22:21:57.615161 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 22:21:57.615684 21623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.599464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41878 (local address 127.29.229.60:46053)
0811 22:21:57.599672 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:57.599676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:57.599718 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:21:57.599977 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:57.599982 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:57.600098 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:57.600241 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:57.601300 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.602558 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.604270 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.604760 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.604879 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.608906 (+ 4027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:57.608911 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:57.608915 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:57.608951 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:57.612211 (+ 3260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.613179 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.613185 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.613189 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.613408 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.613912 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.613918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.613922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.614370 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:57.614543 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:57.615029 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 22:21:57.615310 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 22:21:57.620324 21512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5: Bootstrap starting.
I20260811 22:21:57.623883 21512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:57.625094 21512 log.cc:824] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:57.629735 21512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5: No bootstrap required, opened a new log
I20260811 22:21:57.637686 21512 raft_consensus.cc:348] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } } peers { permanent_uuid: "efc6cc15b503419e9d54b3974204cfd5" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } } peers { permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } }
I20260811 22:21:57.638109 21512 raft_consensus.cc:374] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:57.638302 21512 raft_consensus.cc:729] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6cc15b503419e9d54b3974204cfd5, State: Initialized, Role: FOLLOWER
I20260811 22:21:57.638911 21512 consensus_queue.cc:260] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } } peers { permanent_uuid: "efc6cc15b503419e9d54b3974204cfd5" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } } peers { permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } }
I20260811 22:21:57.641106 21667 sys_catalog.cc:455] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } } peers { permanent_uuid: "efc6cc15b503419e9d54b3974204cfd5" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } } peers { permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } } }
I20260811 22:21:57.641634 21667 sys_catalog.cc:458] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:57.642611 21512 sys_catalog.cc:564] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:21:57.661640 21683 catalog_manager.cc:1558] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:21:57.661823 21683 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:21:57.650455 21400 master_runner.cc:186] Error getting master registration for 127.29.229.62:42551: OK, instance_id { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" instance_seqno: 1786486916241368 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:21:57.663506 21400 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:21:57.674178 21391 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } attempt: 5
W20260811 22:21:57.679457 21550 tablet.cc:2369] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 22:21:57 dist-test-slave-jq4t krb5kdc[21258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486916, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for kudu/127.29.229.60@KRBTEST.COM
I20260811 22:21:57.700273 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.674825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:46053 (local address 127.0.0.1:41894)
0811 22:21:57.675343 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 22:21:57.675367 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:21:57.675499 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:21:57.676371 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:21:57.676377 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:21:57.676981 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:21:57.677202 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:57.677215 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.678758 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.678764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:57.680312 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:21:57.680331 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.681469 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.681476 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:21:57.681666 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.682544 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:21:57.682569 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:21:57.685634 (+ 3065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:21:57.691100 (+ 5466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:57.691106 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:57.691110 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:57.691724 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:57.692221 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:21:57.692226 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:21:57.692230 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:21:57.692443 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:21:57.693387 (+ 944us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:21:57.693394 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:21:57.693581 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 22:21:57.699200 (+ 5619us) client_negotiation.cc:240] Negotiation successful
0811 22:21:57.699700 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368}
I20260811 22:21:57.701776 21623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:21:57.675710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41894 (local address 127.29.229.60:46053)
0811 22:21:57.675886 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:21:57.675892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:21:57.675910 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:21:57.675972 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:21:57.675977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:21:57.676062 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:21:57.676170 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:21:57.677391 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.678589 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.680555 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:21:57.681277 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:21:57.681421 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:21:57.685879 (+ 4458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:21:57.685887 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:21:57.685893 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:21:57.685938 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:21:57.690861 (+ 4923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.691873 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.691879 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.691883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.692058 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:21:57.692678 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:21:57.692684 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:21:57.692687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:21:57.693133 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:21:57.693299 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 22:21:57.700893 (+ 7594us) server_negotiation.cc:299] Negotiation successful
0811 22:21:57.701302 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:21:57.708406 21391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb: Bootstrap starting.
I20260811 22:21:57.717540 21391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb: Neither blocks nor log segments found. Creating new log.
I20260811 22:21:57.719215 21391 log.cc:824] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb: Log is configured to *not* fsync() on all Append() calls
I20260811 22:21:57.724704 21391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb: No bootstrap required, opened a new log
I20260811 22:21:57.736622 21391 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } } peers { permanent_uuid: "efc6cc15b503419e9d54b3974204cfd5" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } } peers { permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } }
I20260811 22:21:57.737829 21391 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:21:57.737958 21391 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d6e7fa36d54a60bea43493a59535cb, State: Initialized, Role: FOLLOWER
I20260811 22:21:57.738793 21391 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } } peers { permanent_uuid: "efc6cc15b503419e9d54b3974204cfd5" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } } peers { permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } }
I20260811 22:21:57.740427 21689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } } peers { permanent_uuid: "efc6cc15b503419e9d54b3974204cfd5" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } } peers { permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } } }
I20260811 22:21:57.740962 21689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:57.742049 21391 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:21:57.742775 21667 raft_consensus.cc:482] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:21:57.743100 21667 raft_consensus.cc:504] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } } peers { permanent_uuid: "efc6cc15b503419e9d54b3974204cfd5" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } } peers { permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } }
I20260811 22:21:57.744949 21667 leader_election.cc:288] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09d6e7fa36d54a60bea43493a59535cb (127.29.229.62:42551), 65da11e950c64f0283ee222ef8073fea (127.29.229.60:46053)
I20260811 22:21:57.746470 21354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "efc6cc15b503419e9d54b3974204cfd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d6e7fa36d54a60bea43493a59535cb" is_pre_election: true
I20260811 22:21:57.747081 21354 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efc6cc15b503419e9d54b3974204cfd5 in term 0.
I20260811 22:21:57.747938 21577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "efc6cc15b503419e9d54b3974204cfd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65da11e950c64f0283ee222ef8073fea" is_pre_election: true
I20260811 22:21:57.748936 21577 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efc6cc15b503419e9d54b3974204cfd5 in term 0.
I20260811 22:21:57.750281 21427 leader_election.cc:302] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65da11e950c64f0283ee222ef8073fea, efc6cc15b503419e9d54b3974204cfd5; no voters:
I20260811 22:21:57.751560 21667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:21:57.751737 21667 raft_consensus.cc:482] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:21:57.751857 21667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:57.755625 21667 raft_consensus.cc:504] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } } peers { permanent_uuid: "efc6cc15b503419e9d54b3974204cfd5" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } } peers { permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } }
I20260811 22:21:57.756963 21354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "efc6cc15b503419e9d54b3974204cfd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d6e7fa36d54a60bea43493a59535cb"
I20260811 22:21:57.757211 21354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:57.763813 21354 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6cc15b503419e9d54b3974204cfd5 in term 1.
I20260811 22:21:57.764424 21428 leader_election.cc:302] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d6e7fa36d54a60bea43493a59535cb, efc6cc15b503419e9d54b3974204cfd5; no voters:
I20260811 22:21:57.768210 21577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "efc6cc15b503419e9d54b3974204cfd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65da11e950c64f0283ee222ef8073fea"
I20260811 22:21:57.768517 21577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:21:57.772526 21577 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6cc15b503419e9d54b3974204cfd5 in term 1.
I20260811 22:21:57.773437 21667 leader_election.cc:288] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [CANDIDATE]: Term 1 election: Requested vote from peers 09d6e7fa36d54a60bea43493a59535cb (127.29.229.62:42551), 65da11e950c64f0283ee222ef8073fea (127.29.229.60:46053)
I20260811 22:21:57.773691 21667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:21:57.773859 21667 raft_consensus.cc:686] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [term 1 LEADER]: Becoming Leader. State: Replica: efc6cc15b503419e9d54b3974204cfd5, State: Running, Role: LEADER
I20260811 22:21:57.774510 21667 consensus_queue.cc:237] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } } peers { permanent_uuid: "efc6cc15b503419e9d54b3974204cfd5" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } } peers { permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } }
I20260811 22:21:57.797856 21691 sys_catalog.cc:455] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efc6cc15b503419e9d54b3974204cfd5. Latest consensus state: current_term: 1 leader_uuid: "efc6cc15b503419e9d54b3974204cfd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } } peers { permanent_uuid: "efc6cc15b503419e9d54b3974204cfd5" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } } peers { permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } } }
I20260811 22:21:57.798494 21691 sys_catalog.cc:458] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:57.810982 21696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:21:57.813784 21696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:21:57.835350 21577 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea [term 1 FOLLOWER]: Refusing update from remote peer efc6cc15b503419e9d54b3974204cfd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:21:57.836427 21691 consensus_queue.cc:1035] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:57.847405 21354 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb [term 1 FOLLOWER]: Refusing update from remote peer efc6cc15b503419e9d54b3974204cfd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260811 22:21:57.848090 21435 tablet.cc:2369] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:21:57.848433 21691 consensus_queue.cc:1035] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:21:57.870543 21689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader efc6cc15b503419e9d54b3974204cfd5. Latest consensus state: current_term: 1 leader_uuid: "efc6cc15b503419e9d54b3974204cfd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } } peers { permanent_uuid: "efc6cc15b503419e9d54b3974204cfd5" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } } peers { permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } } }
I20260811 22:21:57.870954 21689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:57.872466 21643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea [sys.catalog]: SysCatalogTable state changed. Reason: New leader efc6cc15b503419e9d54b3974204cfd5. Latest consensus state: current_term: 1 leader_uuid: "efc6cc15b503419e9d54b3974204cfd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } } peers { permanent_uuid: "efc6cc15b503419e9d54b3974204cfd5" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } } peers { permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } } }
I20260811 22:21:57.872807 21643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:57.877683 21715 mvcc.cc:205] Tried to move back new op lower bound from 7317450415439683584 to 7317450415220600832. Current Snapshot: MvccSnapshot[applied={T|T < 7317450415439683584}]
I20260811 22:21:57.888583 21712 mvcc.cc:205] Tried to move back new op lower bound from 7317450415439683584 to 7317450415220600832. Current Snapshot: MvccSnapshot[applied={T|T < 7317450415439683584}]
I20260811 22:21:57.902858 21667 sys_catalog.cc:455] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "efc6cc15b503419e9d54b3974204cfd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } } peers { permanent_uuid: "efc6cc15b503419e9d54b3974204cfd5" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } } peers { permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } } }
I20260811 22:21:57.903242 21667 sys_catalog.cc:458] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:57.903514 21667 sys_catalog.cc:455] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "efc6cc15b503419e9d54b3974204cfd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } } peers { permanent_uuid: "efc6cc15b503419e9d54b3974204cfd5" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } } peers { permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } } }
I20260811 22:21:57.903754 21667 sys_catalog.cc:458] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5 [sys.catalog]: This master's current role is: LEADER
I20260811 22:21:57.909569 21689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "efc6cc15b503419e9d54b3974204cfd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } } peers { permanent_uuid: "efc6cc15b503419e9d54b3974204cfd5" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } } peers { permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } } }
I20260811 22:21:57.909823 21689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:57.910949 21643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "efc6cc15b503419e9d54b3974204cfd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d6e7fa36d54a60bea43493a59535cb" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42551 } } peers { permanent_uuid: "efc6cc15b503419e9d54b3974204cfd5" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45707 } } peers { permanent_uuid: "65da11e950c64f0283ee222ef8073fea" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 46053 } } }
I20260811 22:21:57.911270 21643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:21:57.919528 21696 catalog_manager.cc:1347] Generated new cluster ID: 1b358cea6ce840b891e54ab6d79186b0
I20260811 22:21:57.919653 21696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260811 22:21:57.931950 21718 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:21:57.932132 21718 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:21:57.952656 21696 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:21:57.954656 21696 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:21:57.973790 21696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efc6cc15b503419e9d54b3974204cfd5: Generated new TSK 0
I20260811 22:21:57.975031 21696 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:21:58.575649 21663 catalog_manager.cc:1259] Loaded cluster ID: 1b358cea6ce840b891e54ab6d79186b0
I20260811 22:21:58.575755 21663 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea: loading cluster ID for follower catalog manager: success
I20260811 22:21:58.578337 21663 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea: acquiring CA information for follower catalog manager: success
I20260811 22:21:58.580132 21663 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 65da11e950c64f0283ee222ef8073fea: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:21:58.934010 21718 catalog_manager.cc:1259] Loaded cluster ID: 1b358cea6ce840b891e54ab6d79186b0
I20260811 22:21:58.934162 21718 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb: loading cluster ID for follower catalog manager: success
I20260811 22:21:58.936785 21718 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb: acquiring CA information for follower catalog manager: success
I20260811 22:21:58.938589 21718 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 09d6e7fa36d54a60bea43493a59535cb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:21:59.140535 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 21281
I20260811 22:21:59.141883 21382 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:21:59.254420 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 21281
I20260811 22:21:59.289947 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 21400
I20260811 22:21:59.291397 21503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:21:59.410492 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 21400
I20260811 22:21:59.448861 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 21523
I20260811 22:21:59.450031 21607 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:21:59.561400 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 21523
2026-08-11T22:21:59Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5581 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:21:59 dist-test-slave-jq4t krb5kdc[21740](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:21:59 dist-test-slave-jq4t krb5kdc[21740](info): set up 2 sockets
Aug 11 22:21:59 dist-test-slave-jq4t krb5kdc[21740](info): commencing operation
krb5kdc: starting...
W20260811 22:22:00.961102 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.346s user 0.006s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:22:01 dist-test-slave-jq4t krb5kdc[21740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:22:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:22:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:22:01.328034 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:37027
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:33897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:37027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:22:01.696674 21759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:01.696995 21759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:01.697090 21759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:01.704779 21759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:22:01.704943 21759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:01.704994 21759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:01.705047 21759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:22:01.705093 21759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:22:01.715441 21759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:33897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:37027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:37027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:01.716846 21759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:01.718163 21759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:01.730726 21766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:01.730755 21769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:01.731060 21765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:01.731623 21759 server_base.cc:1034] running on GCE node
I20260811 22:22:01.732546 21759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:01.734184 21759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:01.735450 21759 hybrid_clock.cc:648] HybridClock initialized: now 1786486921735390 us; error 48 us; skew 500 ppm
I20260811 22:22:01.735860 21759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:01 dist-test-slave-jq4t krb5kdc[21740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486921, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:01.747174 21759 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:22:01.749305 21759 webserver.cc:460] Webserver started at http://127.29.229.62:39431/ using document root <none> and password file <none>
I20260811 22:22:01.750123 21759 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:01.750289 21759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:01.750636 21759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:01.754058 21759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "3f76a5d2a37b4f51962140a27cb257e8"
format_stamp: "Formatted at 2026-08-11 22:22:01 on dist-test-slave-jq4t"
server_key: "b52fa6fb9de9c623f6bc82ef4c731d81"
server_key_iv: "6f82d821acac7a7d01f8a00c9d9d1e3f"
server_key_version: "encryptionkey@0"
I20260811 22:22:01.754866 21759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "3f76a5d2a37b4f51962140a27cb257e8"
format_stamp: "Formatted at 2026-08-11 22:22:01 on dist-test-slave-jq4t"
server_key: "b52fa6fb9de9c623f6bc82ef4c731d81"
server_key_iv: "6f82d821acac7a7d01f8a00c9d9d1e3f"
server_key_version: "encryptionkey@0"
I20260811 22:22:01.760854 21759 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 22:22:01.764830 21776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:01.766471 21759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:22:01.766712 21759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "3f76a5d2a37b4f51962140a27cb257e8"
format_stamp: "Formatted at 2026-08-11 22:22:01 on dist-test-slave-jq4t"
server_key: "b52fa6fb9de9c623f6bc82ef4c731d81"
server_key_iv: "6f82d821acac7a7d01f8a00c9d9d1e3f"
server_key_version: "encryptionkey@0"
I20260811 22:22:01.766943 21759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:01.794427 21759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:01.799259 21759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:01.799603 21759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:01.823864 21759 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:37027
I20260811 22:22:01.823896 21858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:37027 every 8 connection(s)
I20260811 22:22:01.825407 21759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:22:01.829443 21859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:01.830070 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 21759
I20260811 22:22:01.830339 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:22:01.831423 30612 external_mini_cluster.cc:1428] Setting key 9f058cd1b7c3ec09dc96a8c5665937ab
I20260811 22:22:01.845247 21859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8: Bootstrap starting.
Aug 11 22:22:01 dist-test-slave-jq4t krb5kdc[21740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:22:01.849184 21859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:01.850423 21859 log.cc:824] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:01.853587 21859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8: No bootstrap required, opened a new log
I20260811 22:22:01.857064 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:01.833524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51238 (local address 127.29.229.62:37027)
0811 22:22:01.834684 (+ 1160us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:01.834696 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:01.834742 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:22:01.836910 (+ 2168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:01.836942 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:01.837354 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:01.837709 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:01.839705 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:01.841120 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:01.843017 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:01.843681 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:01.843848 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:01.849200 (+ 5352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:01.849208 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:01.849222 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:01.849259 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:01.852724 (+ 3465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:01.853527 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:01.853538 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:01.853551 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:01.853767 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:01.854184 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:01.854190 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:01.854193 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:01.854712 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:01.855040 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:01.855543 (+ 503us) server_negotiation.cc:299] Negotiation successful
0811 22:22:01.855992 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":934,"thread_start_us":709,"threads_started":1}
I20260811 22:22:01.861508 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:22:01.862658 21859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f76a5d2a37b4f51962140a27cb257e8" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37027 } }
I20260811 22:22:01.863085 21859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:01.863204 21859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f76a5d2a37b4f51962140a27cb257e8, State: Initialized, Role: FOLLOWER
I20260811 22:22:01.863785 21859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f76a5d2a37b4f51962140a27cb257e8" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37027 } }
I20260811 22:22:01.864076 21859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:22:01.864234 21859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:22:01.864439 21859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:01.867100 21859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f76a5d2a37b4f51962140a27cb257e8" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37027 } }
I20260811 22:22:01.867679 21859 leader_election.cc:302] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f76a5d2a37b4f51962140a27cb257e8; no voters:
I20260811 22:22:01.868461 21859 leader_election.cc:288] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:22:01.868722 21870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:22:01.869493 21870 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8 [term 1 LEADER]: Becoming Leader. State: Replica: 3f76a5d2a37b4f51962140a27cb257e8, State: Running, Role: LEADER
I20260811 22:22:01.870065 21870 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f76a5d2a37b4f51962140a27cb257e8" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37027 } }
I20260811 22:22:01.870949 21859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:22:01.874035 21871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f76a5d2a37b4f51962140a27cb257e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f76a5d2a37b4f51962140a27cb257e8" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37027 } } }
I20260811 22:22:01.873952 21872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f76a5d2a37b4f51962140a27cb257e8. Latest consensus state: current_term: 1 leader_uuid: "3f76a5d2a37b4f51962140a27cb257e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f76a5d2a37b4f51962140a27cb257e8" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37027 } } }
I20260811 22:22:01.874487 21871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8 [sys.catalog]: This master's current role is: LEADER
I20260811 22:22:01.874501 21872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8 [sys.catalog]: This master's current role is: LEADER
I20260811 22:22:01.876538 21880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:22:01.882045 21880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:22:01.893042 21880 catalog_manager.cc:1347] Generated new cluster ID: 5d731c146d3a4068bb30ebbe6f0780e8
I20260811 22:22:01.893175 21880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:22:01.926057 21880 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:22:01.927474 21880 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:22:01.936465 21880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f76a5d2a37b4f51962140a27cb257e8: Generated new TSK 0
I20260811 22:22:01.937812 21880 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:22:01.976240 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 21759
I20260811 22:22:01.977717 21853 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:02.080714 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 21759
2026-08-11T22:22:02Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2513 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:22:02 dist-test-slave-jq4t krb5kdc[21900](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:22:02 dist-test-slave-jq4t krb5kdc[21900](info): set up 2 sockets
Aug 11 22:22:02 dist-test-slave-jq4t krb5kdc[21900](info): commencing operation
krb5kdc: starting...
W20260811 22:22:03.482755 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.354s user 0.007s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:22:03 dist-test-slave-jq4t krb5kdc[21900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:22:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:22:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:22:03.854303 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:41607
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:41607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:22:04.239673 21916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:04.240084 21916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:04.240269 21916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:04.247985 21916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:22:04.248198 21916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:04.248318 21916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:04.248440 21916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:22:04.248557 21916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:22:04.258944 21916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:41607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:41607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:04.260512 21916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:04.261915 21916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:04.275108 21925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:04.275022 21926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:04.275064 21928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:04.275924 21916 server_base.cc:1034] running on GCE node
I20260811 22:22:04.276836 21916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:04.278362 21916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:04.279582 21916 hybrid_clock.cc:648] HybridClock initialized: now 1786486924279529 us; error 56 us; skew 500 ppm
I20260811 22:22:04.280040 21916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:04 dist-test-slave-jq4t krb5kdc[21900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486924, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:04.283588 21916 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:22:04.285275 21916 webserver.cc:460] Webserver started at http://127.29.229.62:36325/ using document root <none> and password file <none>
I20260811 22:22:04.285988 21916 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:04.286106 21916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:04.286392 21916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:04.289592 21916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "a4a759bd60844b4398243edfdb2dc416"
format_stamp: "Formatted at 2026-08-11 22:22:04 on dist-test-slave-jq4t"
server_key: "73faac4de581783e4e9f593039b75fc2"
server_key_iv: "7f5c596b6b1b68e4e039c52654fd0d38"
server_key_version: "encryptionkey@0"
I20260811 22:22:04.290467 21916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "a4a759bd60844b4398243edfdb2dc416"
format_stamp: "Formatted at 2026-08-11 22:22:04 on dist-test-slave-jq4t"
server_key: "73faac4de581783e4e9f593039b75fc2"
server_key_iv: "7f5c596b6b1b68e4e039c52654fd0d38"
server_key_version: "encryptionkey@0"
I20260811 22:22:04.295979 21916 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:22:04.299964 21937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:04.301669 21916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:22:04.301890 21916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "a4a759bd60844b4398243edfdb2dc416"
format_stamp: "Formatted at 2026-08-11 22:22:04 on dist-test-slave-jq4t"
server_key: "73faac4de581783e4e9f593039b75fc2"
server_key_iv: "7f5c596b6b1b68e4e039c52654fd0d38"
server_key_version: "encryptionkey@0"
I20260811 22:22:04.302125 21916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:04.329209 21916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:04.335362 21916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:04.335732 21916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:04.357586 21916 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:41607
I20260811 22:22:04.357640 22014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:41607 every 8 connection(s)
I20260811 22:22:04.359107 21916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:22:04.361974 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 21916
I20260811 22:22:04.362217 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:22:04.362936 30612 external_mini_cluster.cc:1428] Setting key 59d08667cfab521464b5731a139d75e8
I20260811 22:22:04.363276 22016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:04.379477 22016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416: Bootstrap starting.
Aug 11 22:22:04 dist-test-slave-jq4t krb5kdc[21900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:22:04.383827 22016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:04.385368 22016 log.cc:824] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:04.389806 22016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416: No bootstrap required, opened a new log
I20260811 22:22:04.391767 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:04.365195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41990 (local address 127.29.229.62:41607)
0811 22:22:04.366073 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:04.366089 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:04.366160 (+ 71us) server_negotiation.cc:407] Connection header received
0811 22:22:04.368507 (+ 2347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:04.368536 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:04.369057 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:04.369526 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:04.371037 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:04.373645 (+ 2608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:04.375175 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:04.376329 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:04.376574 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:04.381120 (+ 4546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:04.381134 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:04.381162 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:04.381232 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:04.385762 (+ 4530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:04.387295 (+ 1533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:04.387302 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:04.387309 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:04.387638 (+ 329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:04.388256 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:04.388261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:04.388264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:04.389037 (+ 773us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:04.389367 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:04.389839 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 22:22:04.390330 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":376,"threads_started":1}
I20260811 22:22:04.398806 22016 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a759bd60844b4398243edfdb2dc416" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41607 } }
I20260811 22:22:04.399251 22016 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:04.399379 22016 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4a759bd60844b4398243edfdb2dc416, State: Initialized, Role: FOLLOWER
I20260811 22:22:04.400050 22016 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a759bd60844b4398243edfdb2dc416" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41607 } }
I20260811 22:22:04.400274 22016 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:22:04.400421 22016 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:22:04.400666 22016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:04.403275 22016 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a759bd60844b4398243edfdb2dc416" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41607 } }
I20260811 22:22:04.403797 22016 leader_election.cc:302] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4a759bd60844b4398243edfdb2dc416; no voters:
I20260811 22:22:04.404623 22016 leader_election.cc:288] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:22:04.405045 22026 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:22:04.406461 22026 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 1 LEADER]: Becoming Leader. State: Replica: a4a759bd60844b4398243edfdb2dc416, State: Running, Role: LEADER
I20260811 22:22:04.407274 22016 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:22:04.407140 22026 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a759bd60844b4398243edfdb2dc416" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41607 } }
I20260811 22:22:04.410684 22028 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4a759bd60844b4398243edfdb2dc416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a759bd60844b4398243edfdb2dc416" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41607 } } }
I20260811 22:22:04.410969 22028 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [sys.catalog]: This master's current role is: LEADER
I20260811 22:22:04.410284 22029 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4a759bd60844b4398243edfdb2dc416. Latest consensus state: current_term: 1 leader_uuid: "a4a759bd60844b4398243edfdb2dc416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a759bd60844b4398243edfdb2dc416" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41607 } } }
I20260811 22:22:04.411477 22029 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [sys.catalog]: This master's current role is: LEADER
I20260811 22:22:04.412756 22038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:22:04.418530 22038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:22:04.429144 22038 catalog_manager.cc:1347] Generated new cluster ID: 48c5357d90cf4085969155719877a6fa
I20260811 22:22:04.429402 22038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:22:04.470192 22038 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:22:04.471643 22038 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:22:04.478749 22038 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416: Generated new TSK 0
I20260811 22:22:04.479638 22038 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
I20260811 22:22:04.580861 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41607
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:22:04.970698 22058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:04.971033 22058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:04.971138 22058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:04.979658 22058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:04.979799 22058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:04.979933 22058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:22:04.991529 22058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:04.993326 22058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:04.994700 22058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:05.009610 22063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:05.009649 22064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:05.009577 22069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:05.010718 22058 server_base.cc:1034] running on GCE node
I20260811 22:22:05.011586 22058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:05.012948 22058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:05.014426 22058 hybrid_clock.cc:648] HybridClock initialized: now 1786486925014335 us; error 77 us; skew 500 ppm
I20260811 22:22:05.014853 22058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:05 dist-test-slave-jq4t krb5kdc[21900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486925, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:05.019215 22058 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:22:05.021466 22058 webserver.cc:460] Webserver started at http://127.29.229.1:39505/ using document root <none> and password file <none>
I20260811 22:22:05.022622 22058 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:05.022835 22058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:05.023427 22058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:05.026935 22058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "118b6c918a1c4e4d9531329a84511a08"
format_stamp: "Formatted at 2026-08-11 22:22:05 on dist-test-slave-jq4t"
server_key: "1a7428deff215eb7b5f556b40f9584f9"
server_key_iv: "f9ddc766dabbd003eeef52d6431a8344"
server_key_version: "encryptionkey@0"
I20260811 22:22:05.027890 22058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "118b6c918a1c4e4d9531329a84511a08"
format_stamp: "Formatted at 2026-08-11 22:22:05 on dist-test-slave-jq4t"
server_key: "1a7428deff215eb7b5f556b40f9584f9"
server_key_iv: "f9ddc766dabbd003eeef52d6431a8344"
server_key_version: "encryptionkey@0"
I20260811 22:22:05.034421 22058 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 22:22:05.038573 22078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:05.040191 22058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:22:05.040398 22058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "118b6c918a1c4e4d9531329a84511a08"
format_stamp: "Formatted at 2026-08-11 22:22:05 on dist-test-slave-jq4t"
server_key: "1a7428deff215eb7b5f556b40f9584f9"
server_key_iv: "f9ddc766dabbd003eeef52d6431a8344"
server_key_version: "encryptionkey@0"
I20260811 22:22:05.040685 22058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:05.062012 22058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:05.067754 22058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:05.068112 22058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:05.069397 22058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:05.071094 22058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:05.071202 22058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:05.071362 22058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:05.071437 22058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:05.112089 22058 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:33415
I20260811 22:22:05.112193 22256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:33415 every 8 connection(s)
I20260811 22:22:05.113521 22058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:22:05.120918 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 22058
I20260811 22:22:05.121166 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:22:05.121941 30612 external_mini_cluster.cc:1428] Setting key 305e02f4d50b749d9fdf7c9e25bfaed3
Aug 11 22:22:05 dist-test-slave-jq4t krb5kdc[21900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486925, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:22:05.137984 22260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:05.116239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:53495 (local address 127.29.229.62:41607)
0811 22:22:05.116737 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:05.116745 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:05.117809 (+ 1064us) server_negotiation.cc:407] Connection header received
0811 22:22:05.119641 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:05.119646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:05.119717 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:05.119819 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:05.121683 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.122766 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.125152 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.125862 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.126029 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:05.130476 (+ 4447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:05.130485 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:05.130492 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:05.130542 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:05.134497 (+ 3955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:05.135489 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:05.135495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:05.135499 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:05.135637 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:05.136107 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:05.136116 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:05.136118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:05.136405 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:05.136606 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:05.137357 (+ 751us) server_negotiation.cc:299] Negotiation successful
0811 22:22:05.137606 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":256,"threads_started":1}
I20260811 22:22:05.139065 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:05.116866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:53495)
0811 22:22:05.117646 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:05.117686 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:05.119353 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:05.120111 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:05.120120 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:05.120789 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:05.121490 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:05.121510 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.122972 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.122976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:05.124810 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:05.124825 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.126131 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.126145 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:05.126366 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:05.127133 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:05.127158 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:05.130178 (+ 3020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:05.134693 (+ 4515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:05.134704 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:05.134724 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:05.135346 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:05.135756 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:05.135761 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:05.135765 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:05.135979 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:05.136559 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:05.136567 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:05.136958 (+ 391us) client_negotiation.cc:769] Sending connection context
0811 22:22:05.137242 (+ 284us) client_negotiation.cc:240] Negotiation successful
0811 22:22:05.137818 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":306,"threads_started":1}
I20260811 22:22:05.141167 22257 heartbeater.cc:347] Connected to a master server at 127.29.229.62:41607
I20260811 22:22:05.141532 22257 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:05.142338 22257 heartbeater.cc:511] Master 127.29.229.62:41607 requested a full tablet report, sending...
I20260811 22:22:05.144713 21961 ts_manager.cc:194] Registered new tserver with Master: 118b6c918a1c4e4d9531329a84511a08 (127.29.229.1:33415)
I20260811 22:22:05.146317 21961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:53495
WARNING: no policy specified for kudu/127.29.229.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
I20260811 22:22:05.200330 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41607
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:22:05.574959 22267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:05.575297 22267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:05.575474 22267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:05.583696 22267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:05.583828 22267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:05.584035 22267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:22:05.594455 22267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:05.595842 22267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:05.597363 22267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:05.611840 22272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:05.611984 22273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:05.611958 22277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:05.617859 22267 server_base.cc:1034] running on GCE node
I20260811 22:22:05.618556 22267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:05.619598 22267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:05.620829 22267 hybrid_clock.cc:648] HybridClock initialized: now 1786486925620744 us; error 64 us; skew 500 ppm
I20260811 22:22:05.621196 22267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:05 dist-test-slave-jq4t krb5kdc[21900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486925, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:05.624951 22267 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:22:05.626811 22267 webserver.cc:460] Webserver started at http://127.29.229.2:44775/ using document root <none> and password file <none>
I20260811 22:22:05.627642 22267 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:05.627751 22267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:05.628067 22267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:05.631336 22267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "94def41fac55411786b3a227ee19f217"
format_stamp: "Formatted at 2026-08-11 22:22:05 on dist-test-slave-jq4t"
server_key: "9e13706c38b3236eea78d7a6ba3d2357"
server_key_iv: "ffc4056d3c2bda5706848ad9c6fe43ab"
server_key_version: "encryptionkey@0"
I20260811 22:22:05.632115 22267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "94def41fac55411786b3a227ee19f217"
format_stamp: "Formatted at 2026-08-11 22:22:05 on dist-test-slave-jq4t"
server_key: "9e13706c38b3236eea78d7a6ba3d2357"
server_key_iv: "ffc4056d3c2bda5706848ad9c6fe43ab"
server_key_version: "encryptionkey@0"
I20260811 22:22:05.637705 22267 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260811 22:22:05.641446 22290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:05.642865 22267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:22:05.643054 22267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "94def41fac55411786b3a227ee19f217"
format_stamp: "Formatted at 2026-08-11 22:22:05 on dist-test-slave-jq4t"
server_key: "9e13706c38b3236eea78d7a6ba3d2357"
server_key_iv: "ffc4056d3c2bda5706848ad9c6fe43ab"
server_key_version: "encryptionkey@0"
I20260811 22:22:05.643267 22267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:05.664724 22267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:05.669097 22267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:05.669358 22267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:05.670323 22267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:05.672883 22267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:05.672969 22267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:05.673069 22267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:05.673133 22267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:05.710908 22267 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:35729
I20260811 22:22:05.710935 22446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:35729 every 8 connection(s)
I20260811 22:22:05.712445 22267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:22:05.719502 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 22267
I20260811 22:22:05.719717 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:22:05.720243 30612 external_mini_cluster.cc:1428] Setting key b4395a4612990944c052fd8c9017097d
Aug 11 22:22:05 dist-test-slave-jq4t krb5kdc[21900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486925, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:22:05.736366 22449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:05.715291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:57019 (local address 127.29.229.62:41607)
0811 22:22:05.715750 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:05.715755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:05.716945 (+ 1190us) server_negotiation.cc:407] Connection header received
0811 22:22:05.719033 (+ 2088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:05.719039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:05.719156 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:05.719280 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:05.721231 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.722107 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.723905 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.724715 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.724878 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:05.728736 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:05.728746 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:05.728754 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:05.728811 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:05.732559 (+ 3748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:05.733499 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:05.733507 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:05.733513 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:05.733670 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:05.734162 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:05.734169 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:05.734175 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:05.734599 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:05.734758 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:05.735596 (+ 838us) server_negotiation.cc:299] Negotiation successful
0811 22:22:05.735872 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":241,"threads_started":1}
I20260811 22:22:05.737432 22450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:05.715896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:57019)
0811 22:22:05.716763 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:05.716811 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:05.718728 (+ 1917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:05.719540 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:05.719549 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:05.720220 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:05.721035 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:05.721062 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.722322 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.722327 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:05.723739 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:05.723752 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:05.724950 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:05.724955 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:05.725179 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:05.725922 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:05.725945 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:05.728453 (+ 2508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:05.732720 (+ 4267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:05.732731 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:05.732751 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:05.733352 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:05.733786 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:05.733792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:05.733798 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:05.734036 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:05.734730 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:05.734740 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:05.735171 (+ 431us) client_negotiation.cc:769] Sending connection context
0811 22:22:05.735449 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 22:22:05.736006 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":308,"threads_started":1}
I20260811 22:22:05.739012 22447 heartbeater.cc:347] Connected to a master server at 127.29.229.62:41607
I20260811 22:22:05.739352 22447 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:05.740168 22447 heartbeater.cc:511] Master 127.29.229.62:41607 requested a full tablet report, sending...
I20260811 22:22:05.741946 21961 ts_manager.cc:194] Registered new tserver with Master: 94def41fac55411786b3a227ee19f217 (127.29.229.2:35729)
I20260811 22:22:05.743292 21961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:57019
WARNING: no policy specified for kudu/127.29.229.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
I20260811 22:22:05.795797 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41607
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:22:06.149751 22257 heartbeater.cc:503] Master 127.29.229.62:41607 was elected leader, sending a full tablet report...
W20260811 22:22:06.170059 22453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:06.170374 22453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:06.170475 22453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:06.180104 22453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:06.180250 22453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:06.180373 22453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:22:06.193964 22453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:41607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:06.195864 22453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:06.197657 22453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:06.215003 22460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:06.215108 22462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:06.215000 22464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:06.215961 22453 server_base.cc:1034] running on GCE node
I20260811 22:22:06.216840 22453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:06.217988 22453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:06.219311 22453 hybrid_clock.cc:648] HybridClock initialized: now 1786486926219203 us; error 59 us; skew 500 ppm
I20260811 22:22:06.219905 22453 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:06 dist-test-slave-jq4t krb5kdc[21900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486926, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:06.224718 22453 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:22:06.226987 22453 webserver.cc:460] Webserver started at http://127.29.229.3:38959/ using document root <none> and password file <none>
I20260811 22:22:06.227855 22453 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:06.228012 22453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:06.228377 22453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:06.231827 22453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "b8d7ca4064a1416393d0ac3a25c2f285"
format_stamp: "Formatted at 2026-08-11 22:22:06 on dist-test-slave-jq4t"
server_key: "4c6c4791f3f640c5d4d4c919b26a9ebe"
server_key_iv: "37e13c957a2acd60ecbd03f47574e191"
server_key_version: "encryptionkey@0"
I20260811 22:22:06.232816 22453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "b8d7ca4064a1416393d0ac3a25c2f285"
format_stamp: "Formatted at 2026-08-11 22:22:06 on dist-test-slave-jq4t"
server_key: "4c6c4791f3f640c5d4d4c919b26a9ebe"
server_key_iv: "37e13c957a2acd60ecbd03f47574e191"
server_key_version: "encryptionkey@0"
I20260811 22:22:06.239470 22453 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260811 22:22:06.244212 22472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:06.246258 22453 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 22:22:06.246515 22453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "b8d7ca4064a1416393d0ac3a25c2f285"
format_stamp: "Formatted at 2026-08-11 22:22:06 on dist-test-slave-jq4t"
server_key: "4c6c4791f3f640c5d4d4c919b26a9ebe"
server_key_iv: "37e13c957a2acd60ecbd03f47574e191"
server_key_version: "encryptionkey@0"
I20260811 22:22:06.246734 22453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:06.269222 22453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:06.274024 22453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:06.274363 22453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:06.275501 22453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:06.277141 22453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:06.277261 22453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:06.277429 22453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:06.277508 22453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:06.317407 22453 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:35439
I20260811 22:22:06.317464 22636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:35439 every 8 connection(s)
I20260811 22:22:06.318728 22453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:22:06.327596 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 22453
I20260811 22:22:06.327821 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:22:06.328295 30612 external_mini_cluster.cc:1428] Setting key 66466dbbd9dc6aeffefee3339840b494
Aug 11 22:22:06 dist-test-slave-jq4t krb5kdc[21900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486926, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:22:06.341385 22642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.321363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:36399 (local address 127.29.229.62:41607)
0811 22:22:06.321821 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:06.321834 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:06.322844 (+ 1010us) server_negotiation.cc:407] Connection header received
0811 22:22:06.324769 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:06.324777 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:06.324860 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:06.324958 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:06.326442 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.327366 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.329384 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.329850 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.329999 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.334456 (+ 4457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:06.334462 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:06.334465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:06.334502 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:06.337629 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:06.338780 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:06.338785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:06.338788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:06.338963 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:06.339563 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:06.339567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:06.339569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:06.339885 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:06.340037 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:06.340889 (+ 852us) server_negotiation.cc:299] Negotiation successful
0811 22:22:06.341097 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":222,"threads_started":1}
I20260811 22:22:06.342545 22643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.321942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:36399)
0811 22:22:06.322675 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:06.322715 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:06.324427 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:06.325177 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:06.325185 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:06.325702 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:06.326267 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:06.326284 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.327568 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.327572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:06.329223 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:06.329237 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.330124 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.330133 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:06.330468 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.331284 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:06.331323 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:06.334205 (+ 2882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:06.337909 (+ 3704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:06.337920 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:06.337940 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:06.338588 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:06.339138 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:06.339143 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:06.339146 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:06.339404 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:06.340027 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:06.340034 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:06.340404 (+ 370us) client_negotiation.cc:769] Sending connection context
0811 22:22:06.340757 (+ 353us) client_negotiation.cc:240] Negotiation successful
0811 22:22:06.341220 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":278,"threads_started":1}
I20260811 22:22:06.344038 22639 heartbeater.cc:347] Connected to a master server at 127.29.229.62:41607
I20260811 22:22:06.344374 22639 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:06.345165 22639 heartbeater.cc:511] Master 127.29.229.62:41607 requested a full tablet report, sending...
I20260811 22:22:06.346750 21961 ts_manager.cc:194] Registered new tserver with Master: b8d7ca4064a1416393d0ac3a25c2f285 (127.29.229.3:35439)
I20260811 22:22:06.347735 21961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:36399
I20260811 22:22:06.357214 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:22:06.377524 22642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.362616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36382 (local address 127.29.229.62:41607)
0811 22:22:06.362806 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:06.362815 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:06.363252 (+ 437us) server_negotiation.cc:407] Connection header received
0811 22:22:06.363468 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:06.363476 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:06.363567 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:06.363679 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:06.365223 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.366043 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.367526 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.367922 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.368070 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.370608 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:06.370613 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:06.370616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:06.370648 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:06.373522 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:06.374858 (+ 1336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:06.374867 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:06.374871 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:06.375215 (+ 344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:06.375711 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:06.375717 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:06.375720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:06.376242 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:06.376596 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:06.376944 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 22:22:06.377195 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:22:06.382583 21961 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:22:06.389839 21961 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:22:06.418218 22659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.403375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:35439 (local address 127.0.0.1:43260)
0811 22:22:06.404981 (+ 1606us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:06.405044 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:06.405293 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:06.406658 (+ 1365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:06.406667 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:06.406722 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:06.407206 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:06.407223 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.409833 (+ 2610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.409842 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:06.413244 (+ 3402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:06.413268 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.416674 (+ 3406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.416685 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:06.417045 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.417133 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 22:22:06.417332 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 22:22:06.417621 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1128,"thread_start_us":1429,"threads_started":1}
I20260811 22:22:06.419987 22666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.405124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.0.0.1:44988)
0811 22:22:06.408047 (+ 2923us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:06.408077 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:06.408243 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:06.410136 (+ 1893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:06.410144 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:06.410188 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:06.410845 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:06.410864 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.413556 (+ 2692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.413565 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:06.416738 (+ 3173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:06.416760 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.419120 (+ 2360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.419127 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:06.419348 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.419382 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 22:22:06.419462 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:22:06.419604 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2743,"thread_start_us":13975,"threads_started":1}
I20260811 22:22:06.419996 22660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.404011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43260 (local address 127.29.229.3:35439)
0811 22:22:06.405487 (+ 1476us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:06.405497 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:06.405538 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:22:06.405669 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:06.405679 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:06.405982 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:06.406233 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:06.407463 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.409563 (+ 2100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.413531 (+ 3968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.415654 (+ 2123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.416001 (+ 347us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.416430 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:06.419423 (+ 2993us) server_negotiation.cc:299] Negotiation successful
0811 22:22:06.419625 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1238,"thread_start_us":1420,"threads_started":1}
I20260811 22:22:06.421094 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.405901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44988 (local address 127.29.229.1:33415)
0811 22:22:06.409026 (+ 3125us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:06.409039 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:06.409077 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:22:06.409200 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:06.409212 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:06.409544 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:06.409784 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:06.411164 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.413339 (+ 2175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.417044 (+ 3705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.418817 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.420252 (+ 1435us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.420533 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:06.420610 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 22:22:06.420817 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2894,"thread_start_us":2679,"threads_started":1}
I20260811 22:22:06.424423 22547 tablet_service.cc:1467] Processing CreateTablet for tablet 271d623ec73641adb3d4bf73922a6d0a (DEFAULT_TABLE table=test-table [id=39caba356d9f494db9b3dda69d761795]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:22:06.424654 22160 tablet_service.cc:1467] Processing CreateTablet for tablet 271d623ec73641adb3d4bf73922a6d0a (DEFAULT_TABLE table=test-table [id=39caba356d9f494db9b3dda69d761795]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:22:06.425948 22547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 271d623ec73641adb3d4bf73922a6d0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:06.426115 22160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 271d623ec73641adb3d4bf73922a6d0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:06.427029 22659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.405976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:35729 (local address 127.0.0.1:44670)
0811 22:22:06.418609 (+ 12633us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:06.418637 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:06.418783 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:06.420381 (+ 1598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:06.420386 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:06.420416 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:06.420624 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:06.420636 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.422261 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.422269 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:06.424157 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:06.424174 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.426140 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.426147 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:06.426429 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.426464 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 22:22:06.426573 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 22:22:06.426729 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12488,"thread_start_us":1472,"threads_started":1}
I20260811 22:22:06.427136 22667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.407281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44670 (local address 127.29.229.2:35729)
0811 22:22:06.418293 (+ 11012us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:06.418304 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:06.419196 (+ 892us) server_negotiation.cc:407] Connection header received
0811 22:22:06.419311 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:06.419319 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:06.419608 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:06.419820 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:06.420791 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.422029 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.424319 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.425878 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.426047 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.426263 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:06.426648 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 22:22:06.426836 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10803,"thread_start_us":11657,"threads_started":1}
I20260811 22:22:06.431525 22358 tablet_service.cc:1467] Processing CreateTablet for tablet 271d623ec73641adb3d4bf73922a6d0a (DEFAULT_TABLE table=test-table [id=39caba356d9f494db9b3dda69d761795]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:22:06.433496 22358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 271d623ec73641adb3d4bf73922a6d0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:06.438160 22671 tablet_bootstrap.cc:492] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285: Bootstrap starting.
I20260811 22:22:06.441900 22671 tablet_bootstrap.cc:654] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:06.442811 22672 tablet_bootstrap.cc:492] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08: Bootstrap starting.
I20260811 22:22:06.443343 22671 log.cc:824] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:06.446781 22672 tablet_bootstrap.cc:654] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:06.446971 22671 tablet_bootstrap.cc:492] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285: No bootstrap required, opened a new log
I20260811 22:22:06.447271 22671 ts_tablet_manager.cc:1397] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260811 22:22:06.448796 22672 log.cc:824] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:06.450333 22674 tablet_bootstrap.cc:492] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217: Bootstrap starting.
I20260811 22:22:06.454607 22672 tablet_bootstrap.cc:492] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08: No bootstrap required, opened a new log
I20260811 22:22:06.454924 22672 ts_tablet_manager.cc:1397] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.003s
I20260811 22:22:06.458554 22671 raft_consensus.cc:348] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } }
I20260811 22:22:06.459040 22671 raft_consensus.cc:374] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:06.459141 22671 raft_consensus.cc:729] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d7ca4064a1416393d0ac3a25c2f285, State: Initialized, Role: FOLLOWER
I20260811 22:22:06.459955 22671 consensus_queue.cc:260] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } }
I20260811 22:22:06.464378 22674 tablet_bootstrap.cc:654] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:06.465574 22674 log.cc:824] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:06.468420 22672 raft_consensus.cc:348] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } }
I20260811 22:22:06.469085 22672 raft_consensus.cc:374] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:06.469218 22672 raft_consensus.cc:729] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 118b6c918a1c4e4d9531329a84511a08, State: Initialized, Role: FOLLOWER
I20260811 22:22:06.472023 22674 tablet_bootstrap.cc:492] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217: No bootstrap required, opened a new log
I20260811 22:22:06.472371 22674 ts_tablet_manager.cc:1397] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.000s
I20260811 22:22:06.473333 22672 consensus_queue.cc:260] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } }
I20260811 22:22:06.474828 22639 heartbeater.cc:503] Master 127.29.229.62:41607 was elected leader, sending a full tablet report...
I20260811 22:22:06.475793 22671 ts_tablet_manager.cc:1428] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285: Time spent starting tablet: real 0.028s user 0.013s sys 0.017s
I20260811 22:22:06.477254 22672 ts_tablet_manager.cc:1428] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08: Time spent starting tablet: real 0.022s user 0.014s sys 0.005s
I20260811 22:22:06.481840 22677 raft_consensus.cc:482] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:22:06.482101 22677 raft_consensus.cc:504] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } }
I20260811 22:22:06.487064 22674 raft_consensus.cc:348] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } }
I20260811 22:22:06.487528 22677 leader_election.cc:288] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94def41fac55411786b3a227ee19f217 (127.29.229.2:35729), 118b6c918a1c4e4d9531329a84511a08 (127.29.229.1:33415)
I20260811 22:22:06.487645 22674 raft_consensus.cc:374] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:06.487758 22674 raft_consensus.cc:729] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94def41fac55411786b3a227ee19f217, State: Initialized, Role: FOLLOWER
I20260811 22:22:06.488508 22674 consensus_queue.cc:260] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } }
I20260811 22:22:06.494928 22643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.484146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:35729 (local address 127.29.229.3:54737)
0811 22:22:06.484865 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:06.484892 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:06.485045 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:06.486364 (+ 1319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:06.486372 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:06.486416 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:06.486677 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:06.486694 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.489215 (+ 2521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.489223 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:06.492105 (+ 2882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:06.492125 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.494025 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.494031 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:06.494328 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.494368 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 22:22:06.494472 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 22:22:06.494621 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":207}
I20260811 22:22:06.494903 22447 heartbeater.cc:503] Master 127.29.229.62:41607 was elected leader, sending a full tablet report...
I20260811 22:22:06.495673 22667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.485524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:54737 (local address 127.29.229.2:35729)
0811 22:22:06.485785 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:06.485794 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:06.485817 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:22:06.485907 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:06.485927 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:06.486030 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:06.486143 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:06.486888 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.488825 (+ 1937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.492317 (+ 3492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.493703 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.493839 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.493921 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:06.495232 (+ 1311us) server_negotiation.cc:299] Negotiation successful
0811 22:22:06.495395 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 22:22:06.496002 22674 ts_tablet_manager.cc:1428] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217: Time spent starting tablet: real 0.023s user 0.019s sys 0.001s
I20260811 22:22:06.496773 22384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271d623ec73641adb3d4bf73922a6d0a" candidate_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94def41fac55411786b3a227ee19f217" is_pre_election: true
I20260811 22:22:06.497318 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.484649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:32891 (local address 127.29.229.1:33415)
0811 22:22:06.484858 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:06.484864 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:06.486026 (+ 1162us) server_negotiation.cc:407] Connection header received
0811 22:22:06.486485 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:06.486491 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:06.486584 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:06.486652 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:06.487608 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.488725 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.492261 (+ 3536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.493629 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.493792 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.493871 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:06.496904 (+ 3033us) server_negotiation.cc:299] Negotiation successful
0811 22:22:06.497057 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:22:06.497404 22680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.484372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.29.229.3:32891)
0811 22:22:06.485925 (+ 1553us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:06.485945 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:06.486259 (+ 314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:06.486865 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:06.486870 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:06.486895 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:06.487369 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:06.487383 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.488939 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.488944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:06.492045 (+ 3101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:06.492065 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.494059 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.494065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:06.496715 (+ 2650us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.496743 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:22:06.496868 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 22:22:06.497018 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1424,"mutex_wait_us":2925,"thread_start_us":2883,"threads_started":1}
I20260811 22:22:06.497339 22384 raft_consensus.cc:2454] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8d7ca4064a1416393d0ac3a25c2f285 in term 0.
I20260811 22:22:06.498122 22476 leader_election.cc:302] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94def41fac55411786b3a227ee19f217, b8d7ca4064a1416393d0ac3a25c2f285; no voters:
I20260811 22:22:06.498600 22677 raft_consensus.cc:2781] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:22:06.498764 22677 raft_consensus.cc:482] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:22:06.498875 22677 raft_consensus.cc:3037] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:06.498957 22188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271d623ec73641adb3d4bf73922a6d0a" candidate_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "118b6c918a1c4e4d9531329a84511a08" is_pre_election: true
I20260811 22:22:06.499562 22188 raft_consensus.cc:2454] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8d7ca4064a1416393d0ac3a25c2f285 in term 0.
I20260811 22:22:06.502580 22677 raft_consensus.cc:504] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } }
I20260811 22:22:06.503559 22677 leader_election.cc:288] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [CANDIDATE]: Term 1 election: Requested vote from peers 94def41fac55411786b3a227ee19f217 (127.29.229.2:35729), 118b6c918a1c4e4d9531329a84511a08 (127.29.229.1:33415)
I20260811 22:22:06.505455 22384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271d623ec73641adb3d4bf73922a6d0a" candidate_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94def41fac55411786b3a227ee19f217"
I20260811 22:22:06.505686 22384 raft_consensus.cc:3037] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:06.509212 22188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271d623ec73641adb3d4bf73922a6d0a" candidate_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "118b6c918a1c4e4d9531329a84511a08"
I20260811 22:22:06.509479 22384 raft_consensus.cc:2454] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d7ca4064a1416393d0ac3a25c2f285 in term 1.
I20260811 22:22:06.509477 22188 raft_consensus.cc:3037] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:06.510596 22476 leader_election.cc:302] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94def41fac55411786b3a227ee19f217, b8d7ca4064a1416393d0ac3a25c2f285; no voters:
I20260811 22:22:06.510977 22677 raft_consensus.cc:2781] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:22:06.511513 22677 raft_consensus.cc:686] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [term 1 LEADER]: Becoming Leader. State: Replica: b8d7ca4064a1416393d0ac3a25c2f285, State: Running, Role: LEADER
I20260811 22:22:06.512734 22677 consensus_queue.cc:237] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } }
I20260811 22:22:06.513245 22188 raft_consensus.cc:2454] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d7ca4064a1416393d0ac3a25c2f285 in term 1.
I20260811 22:22:06.520890 21961 catalog_manager.cc:5495] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8d7ca4064a1416393d0ac3a25c2f285 (127.29.229.3). New cstate: current_term: 1 leader_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } health_report { overall_health: UNKNOWN } } }
I20260811 22:22:06.560822 22660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.554958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43262 (local address 127.29.229.3:35439)
0811 22:22:06.555211 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:06.555215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:06.555231 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:22:06.555325 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:06.555328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:06.555421 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:06.555599 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:22:06.556008 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.556850 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.558369 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.558842 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.558984 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.559513 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:22:06.560122 (+ 609us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:22:06.560288 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:06.560394 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 22:22:06.560546 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260811 22:22:06.570338 22640 tablet.cc:2369] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:22:06.570654 22188 raft_consensus.cc:1261] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08 [term 1 FOLLOWER]: Refusing update from remote peer b8d7ca4064a1416393d0ac3a25c2f285: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:22:06.570650 22384 raft_consensus.cc:1261] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217 [term 1 FOLLOWER]: Refusing update from remote peer b8d7ca4064a1416393d0ac3a25c2f285: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:22:06.571798 22684 consensus_queue.cc:1035] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:22:06.572197 22677 consensus_queue.cc:1035] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [LEADER]: Connected to new peer: Peer: permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:22:06.599884 21959 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:22:06.600276 21959 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:22:06.604624 21959 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416: Sending DeleteTablet for 3 replicas of tablet 271d623ec73641adb3d4bf73922a6d0a
I20260811 22:22:06.606016 22547 tablet_service.cc:1514] Processing DeleteTablet for tablet 271d623ec73641adb3d4bf73922a6d0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:22:06 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:43260
I20260811 22:22:06.606035 22358 tablet_service.cc:1514] Processing DeleteTablet for tablet 271d623ec73641adb3d4bf73922a6d0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:22:06 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:44670
I20260811 22:22:06.606045 22160 tablet_service.cc:1514] Processing DeleteTablet for tablet 271d623ec73641adb3d4bf73922a6d0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:22:06 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:44988
I20260811 22:22:06.607422 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 22058
I20260811 22:22:06.621023 22700 tablet_replica.cc:330] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285: stopping tablet replica
I20260811 22:22:06.621536 22700 raft_consensus.cc:2229] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:22:06.622300 22700 raft_consensus.cc:2258] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:22:06.625806 22701 tablet_replica.cc:330] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217: stopping tablet replica
I20260811 22:22:06.626487 22701 raft_consensus.cc:2229] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:22:06.627190 22701 raft_consensus.cc:2258] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:22:06.627260 22700 ts_tablet_manager.cc:1905] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:22:06.631805 22701 ts_tablet_manager.cc:1905] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:22:06.633237 22702 tablet_replica.cc:330] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08: stopping tablet replica
Aug 11 22:22:06 dist-test-slave-jq4t krb5kdc[21900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:22:06.633785 22702 raft_consensus.cc:2229] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:22:06.634657 22702 raft_consensus.cc:2258] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:22:06.638051 22700 ts_tablet_manager.cc:1918] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:22:06.638279 22700 log.cc:1196] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/271d623ec73641adb3d4bf73922a6d0a
I20260811 22:22:06.638928 22700 ts_tablet_manager.cc:1939] T 271d623ec73641adb3d4bf73922a6d0a P b8d7ca4064a1416393d0ac3a25c2f285: Deleting consensus metadata
I20260811 22:22:06.640959 21940 catalog_manager.cc:4841] TS b8d7ca4064a1416393d0ac3a25c2f285 (127.29.229.3:35439): tablet 271d623ec73641adb3d4bf73922a6d0a (table test-table [id=39caba356d9f494db9b3dda69d761795]) successfully deleted
I20260811 22:22:06.642550 22702 ts_tablet_manager.cc:1905] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:22:06.643002 22701 ts_tablet_manager.cc:1918] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:22:06.643265 22701 log.cc:1196] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/271d623ec73641adb3d4bf73922a6d0a
I20260811 22:22:06.643891 22701 ts_tablet_manager.cc:1939] T 271d623ec73641adb3d4bf73922a6d0a P 94def41fac55411786b3a227ee19f217: Deleting consensus metadata
I20260811 22:22:06.644266 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.608723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44994 (local address 127.29.229.1:33415)
0811 22:22:06.608926 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:06.608934 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:06.616620 (+ 7686us) server_negotiation.cc:407] Connection header received
0811 22:22:06.617061 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:06.617067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:06.617164 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:06.617302 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:06.620564 (+ 3262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.621721 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.624270 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.628408 (+ 4138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.628632 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.635235 (+ 6603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:06.635251 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:06.635269 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:06.635326 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:06.639654 (+ 4328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:06.640707 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:06.640717 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:06.640720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:06.640915 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:06.641438 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:06.641442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:06.641445 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:06.641926 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:06.642255 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:06.643558 (+ 1303us) server_negotiation.cc:299] Negotiation successful
0811 22:22:06.643811 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:22:06.645082 22251 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:06.650139 21940 catalog_manager.cc:4841] TS 94def41fac55411786b3a227ee19f217 (127.29.229.2:35729): tablet 271d623ec73641adb3d4bf73922a6d0a (table test-table [id=39caba356d9f494db9b3dda69d761795]) successfully deleted
I20260811 22:22:06.748576 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 22058
W20260811 22:22:06.781149 21942 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.229.1:33415 (error 108)
I20260811 22:22:06.781333 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 22267
I20260811 22:22:06.783866 22659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.783340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.0.0.1:45004)
0811 22:22:06.783520 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:06.783646 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 22:22:06.784185 21942 catalog_manager.cc:4584] TS 118b6c918a1c4e4d9531329a84511a08 (127.29.229.1:33415): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 271d623ec73641adb3d4bf73922a6d0a: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33415: connect: Connection refused (error 111)
Aug 11 22:22:06 dist-test-slave-jq4t krb5kdc[21900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.2@KRBTEST.COM
I20260811 22:22:06.813611 22667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.782955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44678 (local address 127.29.229.2:35729)
0811 22:22:06.783149 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:06.783155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:06.783177 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:22:06.783249 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:06.783255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:06.783347 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:06.783458 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:06.785293 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.786480 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.787934 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.788632 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.788776 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.805505 (+ 16729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:06.805532 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:06.805581 (+ 49us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:06.805641 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:06.810026 (+ 4385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:06.810950 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:06.810961 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:06.810965 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:06.811136 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:06.811712 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:06.811718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:06.811722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:06.812254 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:06.812608 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:06.812979 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 22:22:06.813229 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:22:06.814476 22441 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:06.838317 22659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.837750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.0.0.1:45014)
0811 22:22:06.838017 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:06.838129 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 22:22:06.839515 22659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.839070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.0.0.1:45026)
0811 22:22:06.839236 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:06.839323 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 22:22:06.900193 22659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.899601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.0.0.1:45040)
0811 22:22:06.899872 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:06.899993 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 22:22:06.901503 22659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.901067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.0.0.1:45050)
0811 22:22:06.901230 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:06.901329 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 22:22:06.916679 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 22267
I20260811 22:22:06.947780 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 22453
Aug 11 22:22:06 dist-test-slave-jq4t krb5kdc[21900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.3@KRBTEST.COM
I20260811 22:22:06.969190 22660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.948689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43266 (local address 127.29.229.3:35439)
0811 22:22:06.948877 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:06.948882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:06.948975 (+ 93us) server_negotiation.cc:407] Connection header received
0811 22:22:06.949260 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:06.949270 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:06.949367 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:06.949474 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:06.951320 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.952395 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.954546 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:06.955166 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:06.955348 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:06.961960 (+ 6612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:06.961976 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:06.961997 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:06.962047 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:06.965537 (+ 3490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:06.966622 (+ 1085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:06.966629 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:06.966632 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:06.966830 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:06.967346 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:06.967350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:06.967352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:06.967785 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:06.968033 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:06.968590 (+ 557us) server_negotiation.cc:299] Negotiation successful
0811 22:22:06.968792 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:22:06.970005 22630 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:06.982283 22659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.981706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.0.0.1:45066)
0811 22:22:06.981966 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:06.982082 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 22:22:06.983430 22659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:06.983000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.0.0.1:45078)
0811 22:22:06.983209 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:06.983300 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260811 22:22:07.102593 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 22453
I20260811 22:22:07.134004 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 21916
I20260811 22:22:07.136137 22009 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:07.257514 22659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:07.256965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.0.0.1:45090)
0811 22:22:07.257196 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:07.257336 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 22:22:07.257776 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 21916
I20260811 22:22:07.297472 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:41607
--webserver_interface=127.29.229.62
--webserver_port=36325
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:41607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:22:07.679262 22712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:07.679800 22712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:07.680014 22712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:07.689179 22712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:22:07.689396 22712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:07.689520 22712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:07.689644 22712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:22:07.689729 22712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:22:07.701314 22712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:41607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:41607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=36325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:07.702797 22712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:07.704188 22712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:07.735859 22720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:07.735796 22723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:07.735806 22726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:07.737203 22712 server_base.cc:1034] running on GCE node
I20260811 22:22:07.738154 22712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:07.739945 22712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:07.741185 22712 hybrid_clock.cc:648] HybridClock initialized: now 1786486927741136 us; error 42 us; skew 500 ppm
I20260811 22:22:07.741674 22712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:07 dist-test-slave-jq4t krb5kdc[21900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486927, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:07.746271 22712 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:22:07.749357 22712 webserver.cc:460] Webserver started at http://127.29.229.62:36325/ using document root <none> and password file <none>
I20260811 22:22:07.750536 22712 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:07.750733 22712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:07.759339 22712 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 22:22:07.763602 22737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:07.766019 22712 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 22:22:07.766345 22712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "a4a759bd60844b4398243edfdb2dc416"
format_stamp: "Formatted at 2026-08-11 22:22:04 on dist-test-slave-jq4t"
server_key: "73faac4de581783e4e9f593039b75fc2"
server_key_iv: "7f5c596b6b1b68e4e039c52654fd0d38"
server_key_version: "encryptionkey@0"
I20260811 22:22:07.767766 22712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:07.799263 22712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:07.805327 22712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:07.805795 22712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:07.831518 22712 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:41607
I20260811 22:22:07.831544 22807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:41607 every 8 connection(s)
I20260811 22:22:07.833276 22712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:22:07.837481 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 22712
I20260811 22:22:07.838385 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:33415
--local_ip_for_outbound_sockets=127.29.229.1
--tserver_master_addrs=127.29.229.62:41607
--webserver_port=39505
--webserver_interface=127.29.229.1
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:22:07.843152 22809 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:22:07.847111 22809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416: Bootstrap starting.
I20260811 22:22:07.871138 22809 log.cc:824] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:07.884459 22809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:22:07.885154 22809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416: Bootstrap complete.
I20260811 22:22:07.894879 22809 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a759bd60844b4398243edfdb2dc416" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41607 } }
I20260811 22:22:07.895805 22809 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4a759bd60844b4398243edfdb2dc416, State: Initialized, Role: FOLLOWER
I20260811 22:22:07.896450 22809 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a759bd60844b4398243edfdb2dc416" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41607 } }
I20260811 22:22:07.896790 22809 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:22:07.896932 22809 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:22:07.897130 22809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:22:07.902406 22809 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a759bd60844b4398243edfdb2dc416" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41607 } }
I20260811 22:22:07.903086 22809 leader_election.cc:302] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4a759bd60844b4398243edfdb2dc416; no voters:
I20260811 22:22:07.903847 22809 leader_election.cc:288] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:22:07.904002 22814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:22:07.905560 22814 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 2 LEADER]: Becoming Leader. State: Replica: a4a759bd60844b4398243edfdb2dc416, State: Running, Role: LEADER
I20260811 22:22:07.906162 22814 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a759bd60844b4398243edfdb2dc416" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41607 } }
I20260811 22:22:07.906520 22809 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:22:07.908806 22816 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4a759bd60844b4398243edfdb2dc416. Latest consensus state: current_term: 2 leader_uuid: "a4a759bd60844b4398243edfdb2dc416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a759bd60844b4398243edfdb2dc416" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41607 } } }
I20260811 22:22:07.909237 22816 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [sys.catalog]: This master's current role is: LEADER
I20260811 22:22:07.909862 22815 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a4a759bd60844b4398243edfdb2dc416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a759bd60844b4398243edfdb2dc416" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41607 } } }
I20260811 22:22:07.910197 22815 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [sys.catalog]: This master's current role is: LEADER
I20260811 22:22:07.918387 22825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:22:07.927706 22825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:22:07.929450 22825 catalog_manager.cc:1259] Loaded cluster ID: 48c5357d90cf4085969155719877a6fa
I20260811 22:22:07.929565 22825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:22:07.933030 22825 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 22:22:07.950923 22825 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 22:22:07.961524 22825 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:22:07.963584 22825 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416: Loaded TSK: 0
I20260811 22:22:07.965189 22825 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:22:08.243112 22811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:08.243510 22811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:08.243702 22811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:08.252673 22811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:08.252880 22811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:08.253105 22811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:22:08.266072 22811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:33415
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=39505
--tserver_master_addrs=127.29.229.62:41607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:08.267572 22811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:08.269137 22811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:08.285310 22849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:08.285290 22851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:08.285391 22855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:08.286646 22811 server_base.cc:1034] running on GCE node
I20260811 22:22:08.287393 22811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:08.288414 22811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:08.289708 22811 hybrid_clock.cc:648] HybridClock initialized: now 1786486928289644 us; error 52 us; skew 500 ppm
I20260811 22:22:08.290084 22811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:08 dist-test-slave-jq4t krb5kdc[21900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486928, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:08.293794 22811 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:22:08.295753 22811 webserver.cc:460] Webserver started at http://127.29.229.1:39505/ using document root <none> and password file <none>
I20260811 22:22:08.296907 22811 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:08.297248 22811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:08.303292 22811 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 22:22:08.307284 22865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:08.309080 22811 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:22:08.309280 22811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "118b6c918a1c4e4d9531329a84511a08"
format_stamp: "Formatted at 2026-08-11 22:22:05 on dist-test-slave-jq4t"
server_key: "1a7428deff215eb7b5f556b40f9584f9"
server_key_iv: "f9ddc766dabbd003eeef52d6431a8344"
server_key_version: "encryptionkey@0"
I20260811 22:22:08.310359 22811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:08.330253 22811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:08.336273 22811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:08.336664 22811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:08.338382 22811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:08.341413 22878 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260811 22:22:08.348033 22878 ts_tablet_manager.cc:1852] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260811 22:22:08.349210 22878 ts_tablet_manager.cc:1905] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:22:08.359014 22878 ts_tablet_manager.cc:1918] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:22:08.359308 22878 log.cc:1196] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/wal/wals/271d623ec73641adb3d4bf73922a6d0a
I20260811 22:22:08.359910 22878 ts_tablet_manager.cc:1939] T 271d623ec73641adb3d4bf73922a6d0a P 118b6c918a1c4e4d9531329a84511a08: Deleting consensus metadata
I20260811 22:22:08.360677 22811 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260811 22:22:08.360987 22811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.020s user 0.001s sys 0.000s
I20260811 22:22:08.361215 22811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:08.361332 22811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:08.405128 22811 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:33415
I20260811 22:22:08.405189 23030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:33415 every 8 connection(s)
I20260811 22:22:08.406666 22811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:22:08.409507 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 22811
I20260811 22:22:08.410267 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:35729
--local_ip_for_outbound_sockets=127.29.229.2
--tserver_master_addrs=127.29.229.62:41607
--webserver_port=44775
--webserver_interface=127.29.229.2
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:22:08 dist-test-slave-jq4t krb5kdc[21900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486928, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:22:08.440421 23036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:08.410413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:49567 (local address 127.29.229.62:41607)
0811 22:22:08.411369 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:08.411388 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:08.412040 (+ 652us) server_negotiation.cc:407] Connection header received
0811 22:22:08.415976 (+ 3936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:08.415997 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:08.416670 (+ 673us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:08.417018 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:08.420414 (+ 3396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.421826 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.424875 (+ 3049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.425536 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.425697 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:08.430629 (+ 4932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:08.430637 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:08.430653 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:08.430697 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:08.433760 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:08.434864 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:08.434872 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:08.434877 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:08.435056 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:08.435711 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:08.435718 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:08.435721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:08.436819 (+ 1098us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:08.436998 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:08.438550 (+ 1552us) server_negotiation.cc:299] Negotiation successful
0811 22:22:08.439100 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":521,"threads_started":1}
I20260811 22:22:08.440828 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:08.410630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:49567)
0811 22:22:08.411814 (+ 1184us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:08.411877 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:08.414905 (+ 3028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:08.417462 (+ 2557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:08.417478 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:08.418676 (+ 1198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:08.420014 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:08.420053 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.422047 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.422055 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:08.424624 (+ 2569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:08.424655 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.425806 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.425816 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:08.426151 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:08.427029 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:08.427065 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:08.430304 (+ 3239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:08.433945 (+ 3641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:08.433958 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:08.433979 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:08.434679 (+ 700us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:08.435197 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:08.435206 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:08.435212 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:08.435470 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:08.437085 (+ 1615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:08.437099 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:08.437924 (+ 825us) client_negotiation.cc:769] Sending connection context
0811 22:22:08.438289 (+ 365us) client_negotiation.cc:240] Negotiation successful
0811 22:22:08.438934 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"thread_start_us":577,"threads_started":1}
I20260811 22:22:08.444432 23032 heartbeater.cc:347] Connected to a master server at 127.29.229.62:41607
I20260811 22:22:08.445070 23032 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:08.445895 23032 heartbeater.cc:511] Master 127.29.229.62:41607 requested a full tablet report, sending...
I20260811 22:22:08.448844 22756 ts_manager.cc:194] Registered new tserver with Master: 118b6c918a1c4e4d9531329a84511a08 (127.29.229.1:33415)
I20260811 22:22:08.451318 22756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:49567
W20260811 22:22:08.810335 23039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:08.810673 23039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:08.810779 23039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:08.819672 23039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:08.819797 23039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:08.819924 23039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:22:08.830323 23039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:35729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=44775
--tserver_master_addrs=127.29.229.62:41607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:08.831852 23039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:08.833464 23039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:08.847532 23046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:08.847532 23047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:08.847954 23050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:08.849403 23039 server_base.cc:1034] running on GCE node
I20260811 22:22:08.850072 23039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:08.851116 23039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:08.852375 23039 hybrid_clock.cc:648] HybridClock initialized: now 1786486928852309 us; error 61 us; skew 500 ppm
I20260811 22:22:08.852751 23039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:08 dist-test-slave-jq4t krb5kdc[21900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486928, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:08.856071 23039 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:22:08.857808 23039 webserver.cc:460] Webserver started at http://127.29.229.2:44775/ using document root <none> and password file <none>
I20260811 22:22:08.858615 23039 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:08.858774 23039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:08.864660 23039 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:22:08.867520 23062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:08.869002 23039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:22:08.869185 23039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "94def41fac55411786b3a227ee19f217"
format_stamp: "Formatted at 2026-08-11 22:22:05 on dist-test-slave-jq4t"
server_key: "9e13706c38b3236eea78d7a6ba3d2357"
server_key_iv: "ffc4056d3c2bda5706848ad9c6fe43ab"
server_key_version: "encryptionkey@0"
I20260811 22:22:08.870208 23039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:08.893148 23039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:08.897804 23039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:08.898140 23039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:08.899263 23039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:08.900774 23039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:08.900867 23039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:08.901018 23039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:08.901095 23039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:08.939510 23039 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:35729
I20260811 22:22:08.939769 23218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:35729 every 8 connection(s)
I20260811 22:22:08.941218 23039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:22:08.946944 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 23039
I20260811 22:22:08.947623 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:35439
--local_ip_for_outbound_sockets=127.29.229.3
--tserver_master_addrs=127.29.229.62:41607
--webserver_port=38959
--webserver_interface=127.29.229.3
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:22:08 dist-test-slave-jq4t krb5kdc[21900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486928, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:22:08.966989 23223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:08.944277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:34507 (local address 127.29.229.62:41607)
0811 22:22:08.944832 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:08.944839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:08.945820 (+ 981us) server_negotiation.cc:407] Connection header received
0811 22:22:08.948000 (+ 2180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:08.948008 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:08.948105 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:08.948233 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:08.949870 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.951044 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.952695 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.953509 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.953668 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:08.958104 (+ 4436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:08.958113 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:08.958121 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:08.958170 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:08.962737 (+ 4567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:08.963664 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:08.963669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:08.963673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:08.963973 (+ 300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:08.965040 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:08.965045 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:08.965049 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:08.965383 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:08.965564 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:08.966343 (+ 779us) server_negotiation.cc:299] Negotiation successful
0811 22:22:08.966602 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":247,"threads_started":1}
I20260811 22:22:08.968170 23225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:08.944872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:34507)
0811 22:22:08.945646 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:08.945692 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:08.947653 (+ 1961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:08.948458 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:08.948468 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:08.949078 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:08.949669 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:08.949688 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.951224 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.951228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:08.952557 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:08.952571 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:08.953778 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:08.953784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:08.954000 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:08.954788 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:08.954820 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:08.957784 (+ 2964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:08.962999 (+ 5215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:08.963008 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:08.963024 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:08.963506 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:08.964274 (+ 768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:08.964280 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:08.964285 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:08.964854 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:08.965541 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:08.965550 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:08.965945 (+ 395us) client_negotiation.cc:769] Sending connection context
0811 22:22:08.966239 (+ 294us) client_negotiation.cc:240] Negotiation successful
0811 22:22:08.966832 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":302,"threads_started":1}
I20260811 22:22:08.969841 23219 heartbeater.cc:347] Connected to a master server at 127.29.229.62:41607
I20260811 22:22:08.970181 23219 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:08.970979 23219 heartbeater.cc:511] Master 127.29.229.62:41607 requested a full tablet report, sending...
I20260811 22:22:08.973006 22756 ts_manager.cc:194] Registered new tserver with Master: 94def41fac55411786b3a227ee19f217 (127.29.229.2:35729)
I20260811 22:22:08.974428 22756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:34507
W20260811 22:22:09.318899 23226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:09.319262 23226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:09.319450 23226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:09.327330 23226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:09.327489 23226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:09.327685 23226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:22:09.339035 23226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:35439
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=38959
--tserver_master_addrs=127.29.229.62:41607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:09.340766 23226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:09.342222 23226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:09.356097 23236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:09.356263 23233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:09.356222 23234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:09.357287 23226 server_base.cc:1034] running on GCE node
I20260811 22:22:09.358001 23226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:09.359166 23226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:09.360415 23226 hybrid_clock.cc:648] HybridClock initialized: now 1786486929360340 us; error 62 us; skew 500 ppm
I20260811 22:22:09.360810 23226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:09 dist-test-slave-jq4t krb5kdc[21900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486929, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:09.364869 23226 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:22:09.366640 23226 webserver.cc:460] Webserver started at http://127.29.229.3:38959/ using document root <none> and password file <none>
I20260811 22:22:09.367427 23226 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:09.367609 23226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:09.374154 23226 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:22:09.377476 23249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:09.379150 23226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 22:22:09.379354 23226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "b8d7ca4064a1416393d0ac3a25c2f285"
format_stamp: "Formatted at 2026-08-11 22:22:06 on dist-test-slave-jq4t"
server_key: "4c6c4791f3f640c5d4d4c919b26a9ebe"
server_key_iv: "37e13c957a2acd60ecbd03f47574e191"
server_key_version: "encryptionkey@0"
I20260811 22:22:09.381039 23226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:09.398373 23226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:09.403371 23226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:09.403712 23226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:09.404861 23226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:09.406494 23226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:09.406625 23226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:09.406746 23226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:09.406847 23226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:09.450474 23226 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:35439
I20260811 22:22:09.450572 23407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:35439 every 8 connection(s)
I20260811 22:22:09.452056 23226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:22:09.455046 23032 heartbeater.cc:503] Master 127.29.229.62:41607 was elected leader, sending a full tablet report...
I20260811 22:22:09.458940 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 23226
Aug 11 22:22:09 dist-test-slave-jq4t krb5kdc[21900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486929, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:22:09.476621 23223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.455241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:44729 (local address 127.29.229.62:41607)
0811 22:22:09.455482 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:09.455488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:09.456547 (+ 1059us) server_negotiation.cc:407] Connection header received
0811 22:22:09.458801 (+ 2254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:09.458809 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:09.458969 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:09.459160 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:09.461662 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.462652 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.464601 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.465094 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.465197 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.469689 (+ 4492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:09.469698 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:09.469703 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:09.469747 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:09.473075 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:09.473998 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:09.474011 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:09.474013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:09.474209 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:09.474732 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:09.474737 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:09.474741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:09.475053 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:09.475228 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:09.475995 (+ 767us) server_negotiation.cc:299] Negotiation successful
0811 22:22:09.476240 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:22:09.477566 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.460313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36386 (local address 127.29.229.62:41607)
0811 22:22:09.460845 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:09.460850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:09.462216 (+ 1366us) server_negotiation.cc:407] Connection header received
0811 22:22:09.462325 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:09.462330 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:09.462417 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:09.462560 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:09.464382 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.465425 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.466845 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.467536 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.467685 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.471199 (+ 3514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:09.471208 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:09.471211 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:09.471256 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:09.474641 (+ 3385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:09.475427 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:09.475432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:09.475435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:09.475584 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:09.475993 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:09.475998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:09.476001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:09.476276 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:09.476574 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:09.477013 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 22:22:09.477226 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":303,"threads_started":1}
I20260811 22:22:09.477936 23411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.455478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:44729)
0811 22:22:09.456358 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:09.456414 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:09.458377 (+ 1963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:09.459548 (+ 1171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:09.459557 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:09.460341 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:09.461413 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:09.461445 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.462874 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.462878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:09.464365 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:09.464383 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.465336 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.465343 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:09.465661 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.466409 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:09.466434 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:09.469414 (+ 2980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:09.473216 (+ 3802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:09.473230 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:09.473251 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:09.473744 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:09.474356 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:09.474363 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:09.474367 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:09.474579 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:09.475217 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:09.475226 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:09.475585 (+ 359us) client_negotiation.cc:769] Sending connection context
0811 22:22:09.475903 (+ 318us) client_negotiation.cc:240] Negotiation successful
0811 22:22:09.476501 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":373,"threads_started":1}
I20260811 22:22:09.479377 23408 heartbeater.cc:347] Connected to a master server at 127.29.229.62:41607
I20260811 22:22:09.479722 23408 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:09.480700 23408 heartbeater.cc:511] Master 127.29.229.62:41607 requested a full tablet report, sending...
I20260811 22:22:09.482100 22756 ts_manager.cc:194] Registered new tserver with Master: b8d7ca4064a1416393d0ac3a25c2f285 (127.29.229.3:35439)
I20260811 22:22:09.482858 22756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:44729
I20260811 22:22:09.493253 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:22:09.501092 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.494324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36394 (local address 127.29.229.62:41607)
0811 22:22:09.494525 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:09.494529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:09.495276 (+ 747us) server_negotiation.cc:407] Connection header received
0811 22:22:09.495497 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:09.495501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:09.495647 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:09.495878 (+ 231us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:22:09.496347 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.497111 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.498697 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.499184 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.499300 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.499904 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:22:09.500434 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:22:09.500675 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:09.500809 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 22:22:09.500884 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:22:09.502177 22756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:22:09.510183 22756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:22:09.559048 23424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.526146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.0.0.1:45104)
0811 22:22:09.545416 (+ 19270us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:09.545443 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:09.545617 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:09.546921 (+ 1304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:09.546934 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:09.546979 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:09.547585 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:09.547602 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.552132 (+ 4530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.552141 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:09.555030 (+ 2889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:09.555046 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.557920 (+ 2874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.557930 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:09.558156 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.558210 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 22:22:09.558353 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 22:22:09.558640 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19078,"thread_start_us":17263,"threads_started":1}
I20260811 22:22:09.559067 23426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.526739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45104 (local address 127.29.229.1:33415)
0811 22:22:09.544724 (+ 17985us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:09.544737 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:09.545779 (+ 1042us) server_negotiation.cc:407] Connection header received
0811 22:22:09.545916 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:09.545928 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:09.546297 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:09.546560 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:09.547886 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.551872 (+ 3986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.555234 (+ 3362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.557346 (+ 2112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.557485 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.557746 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:09.558438 (+ 692us) server_negotiation.cc:299] Negotiation successful
0811 22:22:09.558636 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17707,"thread_start_us":18004,"threads_started":1}
I20260811 22:22:09.560830 23428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.543795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:35729 (local address 127.0.0.1:44694)
0811 22:22:09.551707 (+ 7912us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:09.551733 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:09.551910 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:09.553259 (+ 1349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:09.553263 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:09.553292 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:09.553741 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:09.553755 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.555652 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.555658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:09.557782 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:09.557798 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.560027 (+ 2229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.560032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:09.560227 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.560260 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 22:22:09.560362 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:22:09.560514 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7735,"thread_start_us":8225,"threads_started":1}
I20260811 22:22:09.560988 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.543797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44694 (local address 127.29.229.2:35729)
0811 22:22:09.552669 (+ 8872us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:09.552680 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:09.552714 (+ 34us) server_negotiation.cc:407] Connection header received
0811 22:22:09.552835 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:09.552842 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:09.553049 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:09.553281 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:09.553986 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.555488 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.558034 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.559529 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.559660 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.559856 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:09.560444 (+ 588us) server_negotiation.cc:299] Negotiation successful
0811 22:22:09.560658 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8726,"thread_start_us":8584,"threads_started":1}
I20260811 22:22:09.562760 23423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.525727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:35439 (local address 127.0.0.1:43276)
0811 22:22:09.542952 (+ 17225us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:09.543006 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:09.543213 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:09.554469 (+ 11256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:09.554474 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:09.554500 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:09.554916 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:09.554926 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.556811 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.556819 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:09.559022 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:09.559043 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.561845 (+ 2802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.561854 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:09.562036 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.562082 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 22:22:09.562183 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 22:22:09.562327 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16923,"mutex_wait_us":3,"thread_start_us":16738,"threads_started":1}
I20260811 22:22:09.562279 22936 tablet_service.cc:1467] Processing CreateTablet for tablet 563ee9c5199d4c1aa34ff627b89450e8 (DEFAULT_TABLE table=test-table [id=c3a90998414a4af09d24e8d39b53e5de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:22:09.562796 23425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.525972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43276 (local address 127.29.229.3:35439)
0811 22:22:09.554018 (+ 28046us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:09.554024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:09.554050 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:22:09.554121 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:09.554126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:09.554282 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:09.554475 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:09.555121 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.556676 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.559223 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.560653 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.560783 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.561002 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:09.562266 (+ 1264us) server_negotiation.cc:299] Negotiation successful
0811 22:22:09.562451 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27858,"thread_start_us":27635,"threads_started":1}
I20260811 22:22:09.563913 22936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 563ee9c5199d4c1aa34ff627b89450e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:09.565874 23123 tablet_service.cc:1467] Processing CreateTablet for tablet 563ee9c5199d4c1aa34ff627b89450e8 (DEFAULT_TABLE table=test-table [id=c3a90998414a4af09d24e8d39b53e5de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:22:09.568397 23123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 563ee9c5199d4c1aa34ff627b89450e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:09.568063 23312 tablet_service.cc:1467] Processing CreateTablet for tablet 563ee9c5199d4c1aa34ff627b89450e8 (DEFAULT_TABLE table=test-table [id=c3a90998414a4af09d24e8d39b53e5de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:22:09.569722 23312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 563ee9c5199d4c1aa34ff627b89450e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:09.575341 23431 tablet_bootstrap.cc:492] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08: Bootstrap starting.
I20260811 22:22:09.579785 23431 tablet_bootstrap.cc:654] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:09.581284 23431 log.cc:824] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:09.584929 23431 tablet_bootstrap.cc:492] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08: No bootstrap required, opened a new log
I20260811 22:22:09.585326 23431 ts_tablet_manager.cc:1397] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260811 22:22:09.586635 23437 tablet_bootstrap.cc:492] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285: Bootstrap starting.
I20260811 22:22:09.587141 23438 tablet_bootstrap.cc:492] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217: Bootstrap starting.
I20260811 22:22:09.590924 23438 tablet_bootstrap.cc:654] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:09.592103 23437 tablet_bootstrap.cc:654] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:09.592876 23438 log.cc:824] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:09.593458 23437 log.cc:824] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:09.594584 23431 raft_consensus.cc:348] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } }
I20260811 22:22:09.594985 23431 raft_consensus.cc:374] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:09.595111 23431 raft_consensus.cc:729] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 118b6c918a1c4e4d9531329a84511a08, State: Initialized, Role: FOLLOWER
I20260811 22:22:09.595744 23431 consensus_queue.cc:260] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } }
I20260811 22:22:09.596541 23437 tablet_bootstrap.cc:492] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285: No bootstrap required, opened a new log
I20260811 22:22:09.596537 23438 tablet_bootstrap.cc:492] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217: No bootstrap required, opened a new log
I20260811 22:22:09.596834 23438 ts_tablet_manager.cc:1397] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20260811 22:22:09.597451 23431 ts_tablet_manager.cc:1428] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260811 22:22:09.597663 23437 ts_tablet_manager.cc:1397] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260811 22:22:09.605507 23438 raft_consensus.cc:348] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } }
I20260811 22:22:09.605895 23438 raft_consensus.cc:374] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:09.606025 23438 raft_consensus.cc:729] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94def41fac55411786b3a227ee19f217, State: Initialized, Role: FOLLOWER
I20260811 22:22:09.606632 23438 consensus_queue.cc:260] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } }
I20260811 22:22:09.608246 23219 heartbeater.cc:503] Master 127.29.229.62:41607 was elected leader, sending a full tablet report...
I20260811 22:22:09.608109 23437 raft_consensus.cc:348] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } }
I20260811 22:22:09.608680 23437 raft_consensus.cc:374] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:09.608769 23438 ts_tablet_manager.cc:1428] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217: Time spent starting tablet: real 0.012s user 0.010s sys 0.002s
I20260811 22:22:09.608810 23437 raft_consensus.cc:729] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d7ca4064a1416393d0ac3a25c2f285, State: Initialized, Role: FOLLOWER
I20260811 22:22:09.609467 23437 consensus_queue.cc:260] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } }
I20260811 22:22:09.610654 23408 heartbeater.cc:503] Master 127.29.229.62:41607 was elected leader, sending a full tablet report...
I20260811 22:22:09.611956 23437 ts_tablet_manager.cc:1428] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
W20260811 22:22:09.660538 23034 tablet.cc:2369] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:22:09.662396 23443 raft_consensus.cc:482] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:22:09.662899 23443 raft_consensus.cc:504] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } }
I20260811 22:22:09.665441 23443 leader_election.cc:288] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8d7ca4064a1416393d0ac3a25c2f285 (127.29.229.3:35439), 118b6c918a1c4e4d9531329a84511a08 (127.29.229.1:33415)
I20260811 22:22:09.676997 23426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.666195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:59781 (local address 127.29.229.1:33415)
0811 22:22:09.666611 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:09.666622 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:09.667614 (+ 992us) server_negotiation.cc:407] Connection header received
0811 22:22:09.667926 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:09.667933 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:09.668028 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:09.668142 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:09.669376 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.670785 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.674213 (+ 3428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.675539 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.675727 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.675809 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:09.676556 (+ 747us) server_negotiation.cc:299] Negotiation successful
0811 22:22:09.676710 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260811 22:22:09.676988 23425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.666303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:43437 (local address 127.29.229.3:35439)
0811 22:22:09.666610 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:09.666622 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:09.667632 (+ 1010us) server_negotiation.cc:407] Connection header received
0811 22:22:09.667926 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:09.667933 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:09.668027 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:09.668142 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:09.669376 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.670787 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.674212 (+ 3425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.675556 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.675727 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.675809 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:09.676556 (+ 747us) server_negotiation.cc:299] Negotiation successful
0811 22:22:09.676710 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:22:09.677331 23445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.665888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:35439 (local address 127.29.229.2:43437)
0811 22:22:09.667424 (+ 1536us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:09.667444 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:09.667650 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:09.668236 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:09.668242 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:09.668294 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:09.669047 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:09.669057 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.671003 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.671010 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:09.673950 (+ 2940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:09.673971 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.675965 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.675974 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:09.676254 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.676327 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 22:22:09.676441 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 22:22:09.676686 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1301,"mutex_wait_us":342,"spinlock_wait_cycles":20480,"thread_start_us":685,"threads_started":1}
I20260811 22:22:09.677652 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.665923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.29.229.2:59781)
0811 22:22:09.667419 (+ 1496us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:09.667444 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:09.667653 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:09.668236 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:09.668242 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:09.668296 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:09.669032 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:09.669048 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.671013 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.671017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:09.673950 (+ 2933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:09.673971 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.675928 (+ 1957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.675937 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:09.676273 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.676330 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 22:22:09.676441 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 22:22:09.676686 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1356,"mutex_wait_us":368,"spinlock_wait_cycles":66176,"thread_start_us":922,"threads_started":1}
I20260811 22:22:09.678524 23340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "563ee9c5199d4c1aa34ff627b89450e8" candidate_uuid: "94def41fac55411786b3a227ee19f217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" is_pre_election: true
I20260811 22:22:09.678593 22967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "563ee9c5199d4c1aa34ff627b89450e8" candidate_uuid: "94def41fac55411786b3a227ee19f217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "118b6c918a1c4e4d9531329a84511a08" is_pre_election: true
I20260811 22:22:09.679129 23340 raft_consensus.cc:2454] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94def41fac55411786b3a227ee19f217 in term 0.
I20260811 22:22:09.679163 22967 raft_consensus.cc:2454] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94def41fac55411786b3a227ee19f217 in term 0.
I20260811 22:22:09.679903 23065 leader_election.cc:302] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 118b6c918a1c4e4d9531329a84511a08, 94def41fac55411786b3a227ee19f217, b8d7ca4064a1416393d0ac3a25c2f285; no voters:
I20260811 22:22:09.680388 23443 raft_consensus.cc:2781] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:22:09.680583 23443 raft_consensus.cc:482] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:22:09.680716 23443 raft_consensus.cc:3037] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:09.683221 23443 raft_consensus.cc:504] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } }
I20260811 22:22:09.684062 23443 leader_election.cc:288] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [CANDIDATE]: Term 1 election: Requested vote from peers b8d7ca4064a1416393d0ac3a25c2f285 (127.29.229.3:35439), 118b6c918a1c4e4d9531329a84511a08 (127.29.229.1:33415)
I20260811 22:22:09.684546 23340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "563ee9c5199d4c1aa34ff627b89450e8" candidate_uuid: "94def41fac55411786b3a227ee19f217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d7ca4064a1416393d0ac3a25c2f285"
I20260811 22:22:09.684614 22967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "563ee9c5199d4c1aa34ff627b89450e8" candidate_uuid: "94def41fac55411786b3a227ee19f217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "118b6c918a1c4e4d9531329a84511a08"
I20260811 22:22:09.684787 23340 raft_consensus.cc:3037] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:09.684839 22967 raft_consensus.cc:3037] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:09.687124 22967 raft_consensus.cc:2454] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94def41fac55411786b3a227ee19f217 in term 1.
I20260811 22:22:09.687719 23066 leader_election.cc:302] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 118b6c918a1c4e4d9531329a84511a08, 94def41fac55411786b3a227ee19f217; no voters:
I20260811 22:22:09.688064 23443 raft_consensus.cc:2781] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:22:09.688587 23340 raft_consensus.cc:2454] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94def41fac55411786b3a227ee19f217 in term 1.
I20260811 22:22:09.688691 23443 raft_consensus.cc:686] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [term 1 LEADER]: Becoming Leader. State: Replica: 94def41fac55411786b3a227ee19f217, State: Running, Role: LEADER
I20260811 22:22:09.689239 23443 consensus_queue.cc:237] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } }
W20260811 22:22:09.693974 23222 tablet.cc:2369] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:22:09.695271 22755 catalog_manager.cc:5495] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94def41fac55411786b3a227ee19f217 (127.29.229.2). New cstate: current_term: 1 leader_uuid: "94def41fac55411786b3a227ee19f217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94def41fac55411786b3a227ee19f217" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35729 } health_report { overall_health: HEALTHY } } }
W20260811 22:22:09.704032 23409 tablet.cc:2369] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:22:09.737463 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.730136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44704 (local address 127.29.229.2:35729)
0811 22:22:09.730494 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:09.730503 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:09.730537 (+ 34us) server_negotiation.cc:407] Connection header received
0811 22:22:09.730848 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:09.730854 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:09.730997 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:09.731290 (+ 293us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:22:09.731858 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.732914 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.734833 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.735290 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.735455 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.736019 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:22:09.736651 (+ 632us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:22:09.736813 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:09.737045 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 22:22:09.737149 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 22:22:09.746315 22967 raft_consensus.cc:1261] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08 [term 1 FOLLOWER]: Refusing update from remote peer 94def41fac55411786b3a227ee19f217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:22:09.746407 23340 raft_consensus.cc:1261] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285 [term 1 FOLLOWER]: Refusing update from remote peer 94def41fac55411786b3a227ee19f217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:22:09.747606 23443 consensus_queue.cc:1035] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d7ca4064a1416393d0ac3a25c2f285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:22:09.747988 23447 consensus_queue.cc:1035] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "118b6c918a1c4e4d9531329a84511a08" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:22:09.761055 23450 mvcc.cc:205] Tried to move back new op lower bound from 7317450464230076416 to 7317450464011079680. Current Snapshot: MvccSnapshot[applied={T|T < 7317450464230076416}]
I20260811 22:22:09.783131 22756 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:22:09.783440 22756 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:22:09.787545 22756 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416: Sending DeleteTablet for 3 replicas of tablet 563ee9c5199d4c1aa34ff627b89450e8
I20260811 22:22:09.788913 22936 tablet_service.cc:1514] Processing DeleteTablet for tablet 563ee9c5199d4c1aa34ff627b89450e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:22:09 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:45104
I20260811 22:22:09.788954 23123 tablet_service.cc:1514] Processing DeleteTablet for tablet 563ee9c5199d4c1aa34ff627b89450e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:22:09 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:44694
I20260811 22:22:09.789216 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 22811
I20260811 22:22:09.790019 23312 tablet_service.cc:1514] Processing DeleteTablet for tablet 563ee9c5199d4c1aa34ff627b89450e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:22:09 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:43276
I20260811 22:22:09.791363 23464 tablet_replica.cc:330] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217: stopping tablet replica
I20260811 22:22:09.792053 23464 raft_consensus.cc:2229] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:22:09.793126 23464 raft_consensus.cc:2258] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:22:09.795406 23466 tablet_replica.cc:330] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285: stopping tablet replica
I20260811 22:22:09.795794 23466 raft_consensus.cc:2229] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:22:09.796267 23466 raft_consensus.cc:2258] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:22:09.797020 23463 tablet_replica.cc:330] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08: stopping tablet replica
I20260811 22:22:09.797494 23463 raft_consensus.cc:2229] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:22:09.798028 23463 raft_consensus.cc:2258] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:22:09.800122 23466 ts_tablet_manager.cc:1905] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:22:09.800125 23463 ts_tablet_manager.cc:1905] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:22:09.802733 23464 ts_tablet_manager.cc:1905] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:22:09.808102 23463 ts_tablet_manager.cc:1918] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:22:09.808341 23463 log.cc:1196] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/wal/wals/563ee9c5199d4c1aa34ff627b89450e8
I20260811 22:22:09.808985 23426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.790079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45108 (local address 127.29.229.1:33415)
0811 22:22:09.790269 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:09.790278 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:09.790407 (+ 129us) server_negotiation.cc:407] Connection header received
0811 22:22:09.790639 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:09.790647 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:09.790780 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:09.790917 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:09.793184 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.794779 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.796676 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.797408 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.797563 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.801070 (+ 3507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:09.801085 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:09.801102 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:09.801141 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:09.805194 (+ 4053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:09.806108 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:09.806117 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:09.806121 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:09.806372 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:09.806865 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:09.806870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:09.806873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:09.807325 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:09.807757 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:09.808151 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 22:22:09.808451 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:22:09.809031 23463 ts_tablet_manager.cc:1939] T 563ee9c5199d4c1aa34ff627b89450e8 P 118b6c918a1c4e4d9531329a84511a08: Deleting consensus metadata
I20260811 22:22:09.809697 23024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:09.814502 23464 ts_tablet_manager.cc:1918] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:22:09.814751 23464 log.cc:1196] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/563ee9c5199d4c1aa34ff627b89450e8
I20260811 22:22:09.815709 23464 ts_tablet_manager.cc:1939] T 563ee9c5199d4c1aa34ff627b89450e8 P 94def41fac55411786b3a227ee19f217: Deleting consensus metadata
I20260811 22:22:09.816308 23466 ts_tablet_manager.cc:1918] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:22:09.816521 23466 log.cc:1196] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/563ee9c5199d4c1aa34ff627b89450e8
I20260811 22:22:09.817341 23466 ts_tablet_manager.cc:1939] T 563ee9c5199d4c1aa34ff627b89450e8 P b8d7ca4064a1416393d0ac3a25c2f285: Deleting consensus metadata
I20260811 22:22:09.818135 22740 catalog_manager.cc:4841] TS 94def41fac55411786b3a227ee19f217 (127.29.229.2:35729): tablet 563ee9c5199d4c1aa34ff627b89450e8 (table test-table [id=c3a90998414a4af09d24e8d39b53e5de]) successfully deleted
I20260811 22:22:09.819262 22740 catalog_manager.cc:4841] TS b8d7ca4064a1416393d0ac3a25c2f285 (127.29.229.3:35439): tablet 563ee9c5199d4c1aa34ff627b89450e8 (table test-table [id=c3a90998414a4af09d24e8d39b53e5de]) successfully deleted
I20260811 22:22:09.946913 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 22811
W20260811 22:22:09.981530 22741 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.229.1:33415 (error 108)
I20260811 22:22:09.981637 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 23039
I20260811 22:22:09.985351 23423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.984819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.0.0.1:45116)
0811 22:22:09.985026 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:09.985164 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 22:22:09.985769 22741 catalog_manager.cc:4584] TS 118b6c918a1c4e4d9531329a84511a08 (127.29.229.1:33415): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 563ee9c5199d4c1aa34ff627b89450e8: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33415: connect: Connection refused (error 111)
I20260811 22:22:10.000757 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:09.982761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44718 (local address 127.29.229.2:35729)
0811 22:22:09.983000 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:09.983007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:09.983378 (+ 371us) server_negotiation.cc:407] Connection header received
0811 22:22:09.983654 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:09.983661 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:09.983778 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:09.983947 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:09.986062 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.986960 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.988396 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:09.988928 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:09.989083 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:09.992020 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:09.992035 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:09.992053 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:09.992222 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:09.996340 (+ 4118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:09.997619 (+ 1279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:09.997635 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:09.997644 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:09.997868 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:09.998302 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:09.998309 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:09.998317 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:09.998831 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:09.999434 (+ 603us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:09.999865 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 22:22:10.000347 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 22:22:10.001693 23210 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:10.039638 23423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:10.038998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.0.0.1:45126)
0811 22:22:10.039279 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:10.039411 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260811 22:22:10.041294 23423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:10.040617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.0.0.1:45128)
0811 22:22:10.040900 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:10.041082 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 22:22:10.102236 23423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:10.101648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.0.0.1:45132)
0811 22:22:10.101907 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:10.102025 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 22:22:10.103320 23423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:10.102974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.0.0.1:45140)
0811 22:22:10.103118 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:10.103198 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 22:22:10.131475 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 23039
I20260811 22:22:10.164311 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 23226
I20260811 22:22:10.179982 23425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:10.165144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43284 (local address 127.29.229.3:35439)
0811 22:22:10.165344 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:10.165349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:10.165368 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:22:10.165528 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:10.165532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:10.165617 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:10.165702 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:10.167195 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:10.168282 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:10.170002 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:10.170437 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:10.170548 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:10.173535 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:10.173550 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:10.173567 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:10.173601 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:10.177164 (+ 3563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:10.177915 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:10.177923 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:10.177926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:10.178075 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:10.178483 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:10.178486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:10.178488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:10.178865 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:10.179095 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:10.179393 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 22:22:10.179568 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:22:10.180728 23399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:10.183955 23423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:10.183440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.0.0.1:45144)
0811 22:22:10.183629 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:10.183736 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 22:22:10.185470 23423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:10.184839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.0.0.1:45160)
0811 22:22:10.185184 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:10.185316 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260811 22:22:10.276947 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 23226
I20260811 22:22:10.305436 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 22712
I20260811 22:22:10.306663 22801 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:10.430311 23423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:10.429751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33415 (local address 127.0.0.1:45174)
0811 22:22:10.429917 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:10.430054 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 22:22:10.430361 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 22712
I20260811 22:22:10.463582 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:41607
--webserver_interface=127.29.229.62
--webserver_port=36325
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:41607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:22:10.846899 23476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:10.847231 23476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:10.847422 23476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:10.855374 23476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:22:10.855511 23476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:10.855619 23476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:10.855742 23476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:22:10.855824 23476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:22:10.867139 23476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:41607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:41607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=36325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:10.868690 23476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:10.870225 23476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:10.883778 23485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:10.883792 23487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:10.884117 23491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:10.884848 23476 server_base.cc:1034] running on GCE node
I20260811 22:22:10.885769 23476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:10.887404 23476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:10.888618 23476 hybrid_clock.cc:648] HybridClock initialized: now 1786486930888550 us; error 56 us; skew 500 ppm
I20260811 22:22:10.889034 23476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:10 dist-test-slave-jq4t krb5kdc[21900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486930, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:10.892524 23476 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:22:10.894129 23476 webserver.cc:460] Webserver started at http://127.29.229.62:36325/ using document root <none> and password file <none>
I20260811 22:22:10.894846 23476 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:10.894961 23476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:10.900928 23476 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 22:22:10.906605 23505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:10.908135 23476 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 22:22:10.908293 23476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "a4a759bd60844b4398243edfdb2dc416"
format_stamp: "Formatted at 2026-08-11 22:22:04 on dist-test-slave-jq4t"
server_key: "73faac4de581783e4e9f593039b75fc2"
server_key_iv: "7f5c596b6b1b68e4e039c52654fd0d38"
server_key_version: "encryptionkey@0"
I20260811 22:22:10.909418 23476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:10.920899 23476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:10.925235 23476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:10.925484 23476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:10.948073 23476 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:41607
I20260811 22:22:10.948150 23578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:41607 every 8 connection(s)
I20260811 22:22:10.950157 23476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:22:10.951860 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 23476
I20260811 22:22:10.953065 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:33415
--local_ip_for_outbound_sockets=127.29.229.1
--tserver_master_addrs=127.29.229.62:41607
--webserver_port=39505
--webserver_interface=127.29.229.1
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:22:10.963176 23579 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:22:10.968956 23579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416: Bootstrap starting.
I20260811 22:22:10.990909 23579 log.cc:824] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:11.007668 23579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:22:11.008302 23579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416: Bootstrap complete.
I20260811 22:22:11.019373 23579 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a759bd60844b4398243edfdb2dc416" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41607 } }
I20260811 22:22:11.020378 23579 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4a759bd60844b4398243edfdb2dc416, State: Initialized, Role: FOLLOWER
I20260811 22:22:11.021170 23579 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a759bd60844b4398243edfdb2dc416" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41607 } }
I20260811 22:22:11.021416 23579 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:22:11.021548 23579 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:22:11.021759 23579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:22:11.025930 23579 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a759bd60844b4398243edfdb2dc416" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41607 } }
I20260811 22:22:11.026479 23579 leader_election.cc:302] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4a759bd60844b4398243edfdb2dc416; no voters:
I20260811 22:22:11.027287 23579 leader_election.cc:288] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 22:22:11.027448 23585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 22:22:11.028565 23585 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [term 3 LEADER]: Becoming Leader. State: Replica: a4a759bd60844b4398243edfdb2dc416, State: Running, Role: LEADER
I20260811 22:22:11.029186 23585 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a759bd60844b4398243edfdb2dc416" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41607 } }
I20260811 22:22:11.029633 23579 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:22:11.032747 23586 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a4a759bd60844b4398243edfdb2dc416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a759bd60844b4398243edfdb2dc416" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41607 } } }
I20260811 22:22:11.033571 23587 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4a759bd60844b4398243edfdb2dc416. Latest consensus state: current_term: 3 leader_uuid: "a4a759bd60844b4398243edfdb2dc416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a759bd60844b4398243edfdb2dc416" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 41607 } } }
I20260811 22:22:11.033897 23586 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [sys.catalog]: This master's current role is: LEADER
I20260811 22:22:11.033962 23587 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4a759bd60844b4398243edfdb2dc416 [sys.catalog]: This master's current role is: LEADER
I20260811 22:22:11.042268 23593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:22:11.057648 23593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:22:11.059882 23593 catalog_manager.cc:1259] Loaded cluster ID: 48c5357d90cf4085969155719877a6fa
I20260811 22:22:11.060119 23593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 22:22:11.063171 23593 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa9af7e39ad google::LogMessage::Fail() at ??:0
@ 0x7fa9af7e688a google::LogMessage::SendToLog() at ??:0
@ 0x7fa9af7e34de google::LogMessage::Flush() at ??:0
@ 0x7fa9af7e4a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa9c5216512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fa9c5213976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa9c529b610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fa9c529b439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa9b242f345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa9b243291c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fa9b2432799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa9b240deaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa9b56e76db start_thread at ??:0
@ 0x7fa9ae67d61f clone at ??:0
W20260811 22:22:11.410013 23581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:11.410341 23581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:11.410442 23581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:11.417959 23581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:11.418087 23581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:11.418278 23581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:22:11.428828 23581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:33415
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=39505
--tserver_master_addrs=127.29.229.62:41607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:11.430243 23581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:11.431653 23581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:11.450039 23636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:11.450075 23634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:11.450049 23639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:11.451292 23581 server_base.cc:1034] running on GCE node
I20260811 22:22:11.452049 23581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:11.453258 23581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:11.454589 23581 hybrid_clock.cc:648] HybridClock initialized: now 1786486931454514 us; error 47 us; skew 500 ppm
I20260811 22:22:11.455158 23581 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:11 dist-test-slave-jq4t krb5kdc[21900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486931, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:11.459867 23581 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:22:11.462080 23581 webserver.cc:460] Webserver started at http://127.29.229.1:39505/ using document root <none> and password file <none>
I20260811 22:22:11.462927 23581 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:11.463104 23581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:11.469200 23581 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:22:11.472155 23650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:11.474458 23581 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:22:11.474695 23581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "118b6c918a1c4e4d9531329a84511a08"
format_stamp: "Formatted at 2026-08-11 22:22:05 on dist-test-slave-jq4t"
server_key: "1a7428deff215eb7b5f556b40f9584f9"
server_key_iv: "f9ddc766dabbd003eeef52d6431a8344"
server_key_version: "encryptionkey@0"
I20260811 22:22:11.475986 23581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:11.501221 23581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:11.506065 23581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:11.506402 23581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:11.507552 23581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:11.509163 23581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:11.509306 23581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:11.509466 23581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:11.509562 23581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:11.550465 23581 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:33415
I20260811 22:22:11.550558 23812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:33415 every 8 connection(s)
I20260811 22:22:11.552002 23581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:22:11.557354 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 23581
I20260811 22:22:11.558229 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:35729
--local_ip_for_outbound_sockets=127.29.229.2
--tserver_master_addrs=127.29.229.62:41607
--webserver_port=44775
--webserver_interface=127.29.229.2
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:22:11.558291 23819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:11.555589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:38191)
0811 22:22:11.556447 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:11.556965 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":365,"threads_started":1}
W20260811 22:22:11.559527 23817 heartbeater.cc:644] Failed to heartbeat to 127.29.229.62:41607 (0 consecutive failures): Network error: Failed to ping master at 127.29.229.62:41607: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
I20260811 22:22:11.560889 23819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:11.560363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:59107)
0811 22:22:11.560610 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:11.560712 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 22:22:11.561945 23819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:11.561511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:50903)
0811 22:22:11.561680 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:11.561781 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 22:22:11.562486 23817 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:22:11.942591 23820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:11.943347 23820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:11.943538 23820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:11.951823 23820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:11.951952 23820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:11.952170 23820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:22:11.962697 23820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:35729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=44775
--tserver_master_addrs=127.29.229.62:41607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:11.964149 23820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:11.965598 23820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:11.981252 23830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:11.981112 23834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:11.981143 23831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:11.982676 23820 server_base.cc:1034] running on GCE node
I20260811 22:22:11.983438 23820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:11.984649 23820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:11.985929 23820 hybrid_clock.cc:648] HybridClock initialized: now 1786486931985890 us; error 50 us; skew 500 ppm
I20260811 22:22:11.986442 23820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:11 dist-test-slave-jq4t krb5kdc[21900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486931, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:11.994223 23820 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:22:11.996829 23820 webserver.cc:460] Webserver started at http://127.29.229.2:44775/ using document root <none> and password file <none>
I20260811 22:22:11.997946 23820 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:11.998165 23820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:12.006105 23820 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 22:22:12.009486 23846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:12.011281 23820 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:22:12.011514 23820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "94def41fac55411786b3a227ee19f217"
format_stamp: "Formatted at 2026-08-11 22:22:05 on dist-test-slave-jq4t"
server_key: "9e13706c38b3236eea78d7a6ba3d2357"
server_key_iv: "ffc4056d3c2bda5706848ad9c6fe43ab"
server_key_version: "encryptionkey@0"
I20260811 22:22:12.012702 23820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:12.046696 23820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:12.051587 23820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:12.051842 23820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:12.053004 23820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:12.054769 23820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:12.054867 23820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:12.054948 23820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:12.055012 23820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:12.095438 23820 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:35729
I20260811 22:22:12.095561 24009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:35729 every 8 connection(s)
I20260811 22:22:12.097066 23820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:22:12.098306 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 23820
I20260811 22:22:12.099119 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:35439
--local_ip_for_outbound_sockets=127.29.229.3
--tserver_master_addrs=127.29.229.62:41607
--webserver_port=38959
--webserver_interface=127.29.229.3
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:22:12.103273 24012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:12.101241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:39043)
0811 22:22:12.102137 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:12.102381 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":362,"threads_started":1}
W20260811 22:22:12.104137 24010 heartbeater.cc:644] Failed to heartbeat to 127.29.229.62:41607 (0 consecutive failures): Network error: Failed to ping master at 127.29.229.62:41607: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
I20260811 22:22:12.105098 24012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:12.104739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:41921)
0811 22:22:12.104906 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:12.104986 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 22:22:12.106029 24012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:12.105671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:40513)
0811 22:22:12.105819 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:12.105901 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 22:22:12.106240 24010 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:22:12.479274 24014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:12.479671 24014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:12.479840 24014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:12.487502 24014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:12.487628 24014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:12.487816 24014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:22:12.498450 24014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:35439
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=38959
--tserver_master_addrs=127.29.229.62:41607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:12.499835 24014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:12.501312 24014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:12.517128 24023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:12.517176 24025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:12.517689 24027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:12.518900 24014 server_base.cc:1034] running on GCE node
I20260811 22:22:12.519690 24014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:12.520732 24014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:12.522027 24014 hybrid_clock.cc:648] HybridClock initialized: now 1786486932521993 us; error 48 us; skew 500 ppm
I20260811 22:22:12.522459 24014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:12 dist-test-slave-jq4t krb5kdc[21900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486932, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:12.526156 24014 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:22:12.527725 24014 webserver.cc:460] Webserver started at http://127.29.229.3:38959/ using document root <none> and password file <none>
I20260811 22:22:12.528458 24014 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:12.528669 24014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:12.534649 24014 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:22:12.537689 24035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:12.539090 24014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:22:12.539284 24014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "b8d7ca4064a1416393d0ac3a25c2f285"
format_stamp: "Formatted at 2026-08-11 22:22:06 on dist-test-slave-jq4t"
server_key: "4c6c4791f3f640c5d4d4c919b26a9ebe"
server_key_iv: "37e13c957a2acd60ecbd03f47574e191"
server_key_version: "encryptionkey@0"
I20260811 22:22:12.540355 24014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:12.564303 24040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:12.563205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:42399)
0811 22:22:12.563823 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:12.563933 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":335,"threads_started":1}
I20260811 22:22:12.572324 24014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:12.577994 24014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:12.578325 24014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:12.579720 24014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:12.582356 24014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:12.582557 24014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:12.582741 24014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:12.582846 24014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:12.621735 24014 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:35439
I20260811 22:22:12.621797 24190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:35439 every 8 connection(s)
I20260811 22:22:12.623373 24014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:22:12.627537 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 24014
I20260811 22:22:12.628820 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:12.626454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:40399)
0811 22:22:12.627391 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:12.627709 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":333,"threads_started":1}
W20260811 22:22:12.629642 24191 heartbeater.cc:644] Failed to heartbeat to 127.29.229.62:41607 (0 consecutive failures): Network error: Failed to ping master at 127.29.229.62:41607: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
I20260811 22:22:12.630527 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:12.630146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:32769)
0811 22:22:12.630328 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:12.630413 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 22:22:12.631552 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:12.631148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:41319)
0811 22:22:12.631347 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:12.631435 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260811 22:22:12.631845 24191 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:22:13.107906 24196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:13.106970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:59649)
0811 22:22:13.107537 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:13.107649 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":281,"threads_started":1}
I20260811 22:22:13.566103 24198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:13.565349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:59545)
0811 22:22:13.565836 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:13.565925 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":238,"threads_started":1}
I20260811 22:22:13.633459 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:13.632557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:43415)
0811 22:22:13.633065 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:13.633174 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":232,"threads_started":1}
I20260811 22:22:14.110159 24203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:14.109269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:50563)
0811 22:22:14.109879 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:14.109982 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":333,"threads_started":1}
I20260811 22:22:14.568611 24204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:14.567453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:49081)
0811 22:22:14.568188 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:14.568305 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":338,"threads_started":1}
I20260811 22:22:14.635183 24205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:14.634443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:49041)
0811 22:22:14.634918 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:14.635007 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":244,"threads_started":1}
I20260811 22:22:15.112797 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:15.111702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:45845)
0811 22:22:15.112374 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:15.112585 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":355,"threads_started":1}
I20260811 22:22:15.570461 24208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:15.569712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:59589)
0811 22:22:15.570194 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:15.570281 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":253,"threads_started":1}
I20260811 22:22:15.637071 24209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:15.636192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:36879)
0811 22:22:15.636791 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:15.636885 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":382,"threads_started":1}
I20260811 22:22:16.115168 24210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:16.114096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:37039)
0811 22:22:16.114741 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:16.114955 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":325,"threads_started":1}
I20260811 22:22:16.573298 24211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:16.571945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:53777)
0811 22:22:16.572796 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:16.572932 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":379,"threads_started":1}
I20260811 22:22:16.639019 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:16.638213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:46579)
0811 22:22:16.638745 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:16.638835 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":279,"threads_started":1}
I20260811 22:22:17.117223 24214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:17.116346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:60697)
0811 22:22:17.116920 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:17.117008 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":320,"threads_started":1}
I20260811 22:22:17.575479 24215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:17.574549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:54073)
0811 22:22:17.575077 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:17.575285 (+ 208us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":256,"threads_started":1}
I20260811 22:22:17.641062 24216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:17.640166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:36893)
0811 22:22:17.640755 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:17.640880 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":296,"threads_started":1}
I20260811 22:22:18.119096 24218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:18.118248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:43539)
0811 22:22:18.118730 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:18.118916 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":252,"threads_started":1}
I20260811 22:22:18.577468 24220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:18.576617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:39897)
0811 22:22:18.577146 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:18.577231 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":232,"threads_started":1}
I20260811 22:22:18.643169 24221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:18.642227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:35431)
0811 22:22:18.642848 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:18.642984 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":315,"threads_started":1}
I20260811 22:22:19.121161 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:19.120222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:44059)
0811 22:22:19.120842 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:19.120936 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":324,"threads_started":1}
I20260811 22:22:19.579641 24224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:19.578577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:43039)
0811 22:22:19.579278 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:19.579413 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":341,"threads_started":1}
I20260811 22:22:19.645839 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:19.644788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:36703)
0811 22:22:19.645468 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:19.645581 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":350,"threads_started":1}
I20260811 22:22:20.122959 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:20.122160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:40369)
0811 22:22:20.122676 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:20.122763 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":277,"threads_started":1}
I20260811 22:22:20.581670 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:20.580777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:53535)
0811 22:22:20.581378 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:20.581472 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":353,"threads_started":1}
I20260811 22:22:20.648478 24230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:20.647297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:49503)
0811 22:22:20.648095 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:20.648244 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":421,"threads_started":1}
I20260811 22:22:21.125080 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:21.124168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:51423)
0811 22:22:21.124775 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:21.124895 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":348,"threads_started":1}
I20260811 22:22:21.584023 24232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:21.583001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:48383)
0811 22:22:21.583658 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:21.583796 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":350,"threads_started":1}
I20260811 22:22:21.650930 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:21.649695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:49337)
0811 22:22:21.650459 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:21.650678 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":381,"threads_started":1}
I20260811 22:22:22.127434 24234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:22.126475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:53167)
0811 22:22:22.127150 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:22.127258 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":433,"threads_started":1}
I20260811 22:22:22.586763 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:22.585422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:60689)
0811 22:22:22.586292 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:22.586477 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":256,"threads_started":1}
I20260811 22:22:22.653654 24236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:22.652342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:51901)
0811 22:22:22.653292 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:22.653425 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":590,"threads_started":1}
I20260811 22:22:23.130096 24237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:23.128914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:39329)
0811 22:22:23.129682 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:23.129822 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":338,"threads_started":1}
I20260811 22:22:23.588856 24239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:23.588031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:40455)
0811 22:22:23.588586 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:23.588677 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":247,"threads_started":1}
I20260811 22:22:23.655933 24240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:23.654958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:52127)
0811 22:22:23.655551 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:23.655666 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":321,"threads_started":1}
I20260811 22:22:24.133361 24243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:24.131588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:42451)
0811 22:22:24.132961 (+ 1373us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:24.133094 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1201,"thread_start_us":1028,"threads_started":1}
I20260811 22:22:24.591519 24244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:24.590286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:37711)
0811 22:22:24.591166 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:24.591296 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":499,"threads_started":1}
I20260811 22:22:24.657730 24245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:24.656964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:38215)
0811 22:22:24.657483 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:24.657570 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":287,"threads_started":1}
I20260811 22:22:25.135357 24246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:25.134433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:53477)
0811 22:22:25.135024 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:25.135138 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":262,"threads_started":1}
I20260811 22:22:25.594266 24247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:25.592934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:45669)
0811 22:22:25.593985 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:25.594070 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":920,"thread_start_us":801,"threads_started":1}
I20260811 22:22:25.659758 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:25.658803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:40719)
0811 22:22:25.659434 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:25.659542 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":295,"threads_started":1}
I20260811 22:22:26.137328 24250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:26.136549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:46745)
0811 22:22:26.137045 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:26.137134 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":247,"threads_started":1}
I20260811 22:22:26.596276 24251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:26.595442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:52067)
0811 22:22:26.595954 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:26.596073 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":250,"threads_started":1}
I20260811 22:22:26.661741 24253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:26.660873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:36447)
0811 22:22:26.661406 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:26.661506 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":287,"threads_started":1}
I20260811 22:22:27.139533 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:27.138707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.2:60157)
0811 22:22:27.139241 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:27.139337 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":304,"threads_started":1}
I20260811 22:22:27.598042 24255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:27.597293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.1:41367)
0811 22:22:27.597769 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:27.597868 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":252,"threads_started":1}
I20260811 22:22:27.636292 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 23581
I20260811 22:22:27.655514 24257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:27.637619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40032 (local address 127.29.229.1:33415)
0811 22:22:27.638543 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:27.638558 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:27.638603 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:22:27.640243 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:27.640252 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:27.640757 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:27.641070 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:27.642479 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:27.643751 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:27.645200 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:27.645910 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:27.646046 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:27.648962 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:27.648970 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:27.648985 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:27.649029 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:27.652117 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:27.652994 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:27.653000 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:27.653007 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:27.653196 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:27.653599 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:27.653603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:27.653605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:27.654175 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:27.654431 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:27.654868 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 22:22:27.655211 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719,"thread_start_us":348,"threads_started":1}
I20260811 22:22:27.656968 23804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:27.663774 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:27.662938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:41607 (local address 127.29.229.3:35657)
0811 22:22:27.663456 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:27.663589 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:41607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":276,"threads_started":1}
I20260811 22:22:27.744262 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 23581
I20260811 22:22:27.776037 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 23820
I20260811 22:22:27.797425 24262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:27.777293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54368 (local address 127.29.229.2:35729)
0811 22:22:27.777892 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:27.777908 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:27.777949 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:22:27.779657 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:27.779668 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:27.780113 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:27.780436 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:27.782069 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:27.783354 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:27.785017 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:27.785750 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:27.786036 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:27.790059 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:27.790068 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:27.790087 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:27.790145 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:27.793677 (+ 3532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:27.794659 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:27.794666 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:27.794674 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:27.794869 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:27.795275 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:27.795279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:27.795281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:27.795850 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:27.796114 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:27.796648 (+ 534us) server_negotiation.cc:299] Negotiation successful
0811 22:22:27.797095 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":269,"threads_started":1}
I20260811 22:22:27.798976 24002 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:27.887763 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 23820
I20260811 22:22:27.919868 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 24014
I20260811 22:22:27.939213 24264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:27.921365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60136 (local address 127.29.229.3:35439)
0811 22:22:27.922046 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:27.922062 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:27.922120 (+ 58us) server_negotiation.cc:407] Connection header received
0811 22:22:27.923863 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:27.923874 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:27.924323 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:27.924749 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:27.926349 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:27.927596 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:27.928953 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:27.929631 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:27.929843 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:27.932608 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:27.932617 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:27.932631 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:27.932682 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:27.935606 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:27.936367 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:27.936376 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:27.936386 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:27.936636 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:27.937050 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:27.937056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:27.937059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:27.937629 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:27.937989 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:27.938443 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 22:22:27.938897 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":317,"threads_started":1}
I20260811 22:22:27.940881 24185 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:28.029835 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 24014
2026-08-11T22:22:28Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25959 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:22:28 dist-test-slave-jq4t krb5kdc[24272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:22:28 dist-test-slave-jq4t krb5kdc[24272](info): set up 2 sockets
Aug 11 22:22:28 dist-test-slave-jq4t krb5kdc[24272](info): commencing operation
krb5kdc: starting...
W20260811 22:22:29.475514 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.387s user 0.003s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:22:29 dist-test-slave-jq4t krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:22:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:22:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:22:29.854295 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:45475
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:45475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:22:30.237625 24295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:30.238011 24295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:30.238204 24295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:30.246311 24295 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:22:30.246482 24295 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:22:30.246574 24295 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:22:30.246676 24295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:22:30.246781 24295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:30.247169 24295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:30.247334 24295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:22:30.247443 24295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:22:30.258800 24295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:45475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:45475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:30.260275 24295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:30.261722 24295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:30.276122 24309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:30.276157 24305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:30.276376 24303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:30.277211 24295 server_base.cc:1034] running on GCE node
I20260811 22:22:30.278119 24295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:30.279613 24295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:30.280901 24295 hybrid_clock.cc:648] HybridClock initialized: now 1786486950280799 us; error 73 us; skew 500 ppm
I20260811 22:22:30.281391 24295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:30 dist-test-slave-jq4t krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486950, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:30.285073 24295 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:22:30.286860 24295 webserver.cc:460] Webserver started at http://127.29.229.62:41253/ using document root <none> and password file <none>
I20260811 22:22:30.287642 24295 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:30.287788 24295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:30.288144 24295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:30.291500 24295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6"
format_stamp: "Formatted at 2026-08-11 22:22:30 on dist-test-slave-jq4t"
server_key: "b7aa420b88eac14b438c66c7ce86e61b"
server_key_iv: "90b12d4a88f80c6e71b51e48f4dab6db"
server_key_version: "encryptionkey@0"
I20260811 22:22:30.292898 24295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6"
format_stamp: "Formatted at 2026-08-11 22:22:30 on dist-test-slave-jq4t"
server_key: "b7aa420b88eac14b438c66c7ce86e61b"
server_key_iv: "90b12d4a88f80c6e71b51e48f4dab6db"
server_key_version: "encryptionkey@0"
I20260811 22:22:30.298599 24295 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:22:30.302350 24318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:30.303776 24295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:22:30.303970 24295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6"
format_stamp: "Formatted at 2026-08-11 22:22:30 on dist-test-slave-jq4t"
server_key: "b7aa420b88eac14b438c66c7ce86e61b"
server_key_iv: "90b12d4a88f80c6e71b51e48f4dab6db"
server_key_version: "encryptionkey@0"
I20260811 22:22:30.304215 24295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:30.338788 24295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:30.343246 24295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:30.343640 24295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:30.365648 24295 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:45475
I20260811 22:22:30.365675 24392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:45475 every 8 connection(s)
I20260811 22:22:30.367141 24295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:22:30.371245 24395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:30.372339 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 24295
I20260811 22:22:30.372609 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:22:30.373353 30612 external_mini_cluster.cc:1428] Setting key 9d806821a2c0eb6169a64cede4accc31
I20260811 22:22:30.384757 24395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6: Bootstrap starting.
I20260811 22:22:30.390544 24395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6: Neither blocks nor log segments found. Creating new log.
Aug 11 22:22:30 dist-test-slave-jq4t krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:22:30.392002 24395 log.cc:824] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:30.395598 24395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6: No bootstrap required, opened a new log
I20260811 22:22:30.402957 24401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:30.375899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47976 (local address 127.29.229.62:45475)
0811 22:22:30.376988 (+ 1089us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:30.377001 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:30.377051 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:22:30.378866 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:30.378893 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:30.379437 (+ 544us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:30.379799 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:30.381897 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:30.383524 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:30.385564 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:30.386312 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:30.386531 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:30.393173 (+ 6642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:30.393188 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:30.393215 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:30.393270 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:30.396844 (+ 3574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:30.397953 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:30.397960 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:30.397967 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:30.398153 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:30.398820 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:30.398826 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:30.398829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:30.399834 (+ 1005us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:30.400191 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:30.401195 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0811 22:22:30.401736 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":824,"thread_start_us":464,"threads_started":1}
I20260811 22:22:30.405702 24395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45475 } }
I20260811 22:22:30.406222 24395 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:30.406344 24395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a3ecbba44a646a1bba5bb4f0fa56ba6, State: Initialized, Role: FOLLOWER
I20260811 22:22:30.407018 24395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45475 } }
I20260811 22:22:30.407306 24395 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:22:30.407464 24395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:22:30.407655 24395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:30.410243 24395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45475 } }
I20260811 22:22:30.410905 24395 leader_election.cc:302] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a3ecbba44a646a1bba5bb4f0fa56ba6; no voters:
I20260811 22:22:30.411823 24395 leader_election.cc:288] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:22:30.411985 24403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:22:30.412758 24403 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 1 LEADER]: Becoming Leader. State: Replica: 6a3ecbba44a646a1bba5bb4f0fa56ba6, State: Running, Role: LEADER
I20260811 22:22:30.413378 24403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45475 } }
I20260811 22:22:30.414338 24395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:22:30.416774 24405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45475 } } }
I20260811 22:22:30.417574 24405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:22:30.416873 24408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a3ecbba44a646a1bba5bb4f0fa56ba6. Latest consensus state: current_term: 1 leader_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45475 } } }
I20260811 22:22:30.421326 24408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:22:30.424973 24417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:22:30.430783 24417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:22:30.439944 24417 catalog_manager.cc:1347] Generated new cluster ID: a5362d30f0074d67a133a754b4584d98
I20260811 22:22:30.440120 24417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:22:30.457988 24417 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:22:30.459969 24417 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:22:30.470290 24417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6: Generated new TSK 0
I20260811 22:22:30.471443 24417 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
I20260811 22:22:30.558311 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:45475
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:22:30.942245 24437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:30.942574 24437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:30.942749 24437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:30.950440 24437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:30.950583 24437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:30.950817 24437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:22:30.961066 24437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:45475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:30.962471 24437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:30.963821 24437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:30.977562 24443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:30.977635 24444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:30.977674 24447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:30.978631 24437 server_base.cc:1034] running on GCE node
I20260811 22:22:30.979312 24437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:30.980533 24437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:30.981770 24437 hybrid_clock.cc:648] HybridClock initialized: now 1786486950981719 us; error 61 us; skew 500 ppm
I20260811 22:22:30.982229 24437 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:30 dist-test-slave-jq4t krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486950, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:30.985821 24437 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:22:30.987450 24437 webserver.cc:460] Webserver started at http://127.29.229.1:38211/ using document root <none> and password file <none>
I20260811 22:22:30.988149 24437 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:30.988260 24437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:30.988700 24437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:30.991782 24437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "454fa69cf4d844b79d7688db9d8e722b"
format_stamp: "Formatted at 2026-08-11 22:22:30 on dist-test-slave-jq4t"
server_key: "e0f3998d51d997339b594aaa503b1e17"
server_key_iv: "6f9bef3f96de25cd0a983527762693dd"
server_key_version: "encryptionkey@0"
I20260811 22:22:30.992663 24437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "454fa69cf4d844b79d7688db9d8e722b"
format_stamp: "Formatted at 2026-08-11 22:22:30 on dist-test-slave-jq4t"
server_key: "e0f3998d51d997339b594aaa503b1e17"
server_key_iv: "6f9bef3f96de25cd0a983527762693dd"
server_key_version: "encryptionkey@0"
I20260811 22:22:30.998761 24437 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 22:22:31.003245 24456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:31.005425 24437 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 22:22:31.005699 24437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "454fa69cf4d844b79d7688db9d8e722b"
format_stamp: "Formatted at 2026-08-11 22:22:30 on dist-test-slave-jq4t"
server_key: "e0f3998d51d997339b594aaa503b1e17"
server_key_iv: "6f9bef3f96de25cd0a983527762693dd"
server_key_version: "encryptionkey@0"
I20260811 22:22:31.005990 24437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:31.023149 24437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:31.029628 24437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:31.030019 24437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:31.031198 24437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:31.032788 24437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:31.032904 24437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:31.033124 24437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:31.033209 24437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:31.073627 24437 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:33947
I20260811 22:22:31.073690 24615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:33947 every 8 connection(s)
I20260811 22:22:31.074993 24437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:22:31.077550 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 24437
I20260811 22:22:31.077776 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:22:31.078691 30612 external_mini_cluster.cc:1428] Setting key cad9b3a77bf3bd19b17360807a11343d
Aug 11 22:22:31 dist-test-slave-jq4t krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486950, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:22:31.100849 24620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:31.078225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:45681 (local address 127.29.229.62:45475)
0811 22:22:31.078799 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:31.078806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:31.080068 (+ 1262us) server_negotiation.cc:407] Connection header received
0811 22:22:31.082117 (+ 2049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:31.082125 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:31.082217 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:31.082350 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:31.083909 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:31.085092 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:31.086790 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:31.087526 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:31.087677 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:31.092613 (+ 4936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:31.092619 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:31.092622 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:31.092657 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:31.096264 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:31.097557 (+ 1293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:31.097563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:31.097568 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:31.097735 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:31.098328 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:31.098340 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:31.098343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:31.098685 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:31.098875 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:31.099841 (+ 966us) server_negotiation.cc:299] Negotiation successful
0811 22:22:31.100317 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":294,"threads_started":1}
I20260811 22:22:31.102241 24621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:31.078803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:45681)
0811 22:22:31.079871 (+ 1068us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:31.079947 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:31.081836 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:31.082564 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:31.082572 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:31.083107 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:31.083723 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:31.083741 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:31.085308 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:31.085314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:31.086631 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:31.086645 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:31.087796 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:31.087805 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:31.088199 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:31.089062 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:31.089095 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:31.092306 (+ 3211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:31.096648 (+ 4342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:31.096660 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:31.096682 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:31.097353 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:31.097879 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:31.097886 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:31.097890 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:31.098179 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:31.098841 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:31.098852 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:31.099362 (+ 510us) client_negotiation.cc:769] Sending connection context
0811 22:22:31.099728 (+ 366us) client_negotiation.cc:240] Negotiation successful
0811 22:22:31.100705 (+ 977us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":380,"threads_started":1}
I20260811 22:22:31.104629 24616 heartbeater.cc:347] Connected to a master server at 127.29.229.62:45475
I20260811 22:22:31.105053 24616 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:31.106101 24616 heartbeater.cc:511] Master 127.29.229.62:45475 requested a full tablet report, sending...
I20260811 22:22:31.109261 24340 ts_manager.cc:194] Registered new tserver with Master: 454fa69cf4d844b79d7688db9d8e722b (127.29.229.1:33947)
I20260811 22:22:31.111124 24340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:45681
WARNING: no policy specified for kudu/127.29.229.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
I20260811 22:22:31.160367 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:45475
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:22:31.533996 24625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:31.534301 24625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:31.534402 24625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:31.541975 24625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:31.542098 24625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:31.542214 24625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:22:31.552944 24625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:45475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:31.554241 24625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:31.555536 24625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:31.569361 24633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:31.569299 24634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:31.569326 24636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:31.570472 24625 server_base.cc:1034] running on GCE node
I20260811 22:22:31.571095 24625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:31.572064 24625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:31.573331 24625 hybrid_clock.cc:648] HybridClock initialized: now 1786486951573271 us; error 41 us; skew 500 ppm
I20260811 22:22:31.573701 24625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:31 dist-test-slave-jq4t krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486951, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:31.577239 24625 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:22:31.578811 24625 webserver.cc:460] Webserver started at http://127.29.229.2:44573/ using document root <none> and password file <none>
I20260811 22:22:31.579543 24625 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:31.579685 24625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:31.580025 24625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:31.583122 24625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "6b9d9dfba06945f88970c30debaee5ae"
format_stamp: "Formatted at 2026-08-11 22:22:31 on dist-test-slave-jq4t"
server_key: "0b4fa2daba7d77bed1e8c60374d378d9"
server_key_iv: "59379c81bfd6c61a184b79d6ee436e13"
server_key_version: "encryptionkey@0"
I20260811 22:22:31.583904 24625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "6b9d9dfba06945f88970c30debaee5ae"
format_stamp: "Formatted at 2026-08-11 22:22:31 on dist-test-slave-jq4t"
server_key: "0b4fa2daba7d77bed1e8c60374d378d9"
server_key_iv: "59379c81bfd6c61a184b79d6ee436e13"
server_key_version: "encryptionkey@0"
I20260811 22:22:31.589564 24625 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 22:22:31.593225 24647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:31.594885 24625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 22:22:31.595082 24625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "6b9d9dfba06945f88970c30debaee5ae"
format_stamp: "Formatted at 2026-08-11 22:22:31 on dist-test-slave-jq4t"
server_key: "0b4fa2daba7d77bed1e8c60374d378d9"
server_key_iv: "59379c81bfd6c61a184b79d6ee436e13"
server_key_version: "encryptionkey@0"
I20260811 22:22:31.595293 24625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:31.609699 24625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:31.617544 24625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:31.617872 24625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:31.618927 24625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:31.620308 24625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:31.620419 24625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:31.620594 24625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:31.620674 24625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:31.656734 24625 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:39553
I20260811 22:22:31.656795 24813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:39553 every 8 connection(s)
I20260811 22:22:31.658138 24625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:22:31.658864 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 24625
I20260811 22:22:31.659461 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:22:31.660203 30612 external_mini_cluster.cc:1428] Setting key 216588f090575d94fbc2ec295ef952f3
Aug 11 22:22:31 dist-test-slave-jq4t krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486951, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:22:31.681139 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:31.661473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:46127 (local address 127.29.229.62:45475)
0811 22:22:31.661984 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:31.661991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:31.663188 (+ 1197us) server_negotiation.cc:407] Connection header received
0811 22:22:31.665221 (+ 2033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:31.665226 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:31.665323 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:31.665455 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:31.667052 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:31.667818 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:31.669498 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:31.669939 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:31.670077 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:31.674376 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:31.674381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:31.674385 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:31.674417 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:31.677740 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:31.678597 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:31.678602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:31.678608 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:31.678738 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:31.679216 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:31.679225 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:31.679227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:31.679517 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:31.679687 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:31.680430 (+ 743us) server_negotiation.cc:299] Negotiation successful
0811 22:22:31.680718 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":270,"threads_started":1}
I20260811 22:22:31.682209 24820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:31.662102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:46127)
0811 22:22:31.663028 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:31.663068 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:31.664943 (+ 1875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:31.665725 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:31.665734 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:31.666277 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:31.666876 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:31.666894 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:31.667991 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:31.667995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:31.669348 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:31.669361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:31.670181 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:31.670189 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:31.670429 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:31.671313 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:31.671346 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:31.674119 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:31.677950 (+ 3831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:31.677967 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:31.677983 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:31.678456 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:31.678862 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:31.678867 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:31.678870 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:31.679083 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:31.679658 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:31.679667 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:31.680048 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 22:22:31.680327 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 22:22:31.680966 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":386,"threads_started":1}
I20260811 22:22:31.683869 24815 heartbeater.cc:347] Connected to a master server at 127.29.229.62:45475
I20260811 22:22:31.684233 24815 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:31.685114 24815 heartbeater.cc:511] Master 127.29.229.62:45475 requested a full tablet report, sending...
I20260811 22:22:31.686872 24340 ts_manager.cc:194] Registered new tserver with Master: 6b9d9dfba06945f88970c30debaee5ae (127.29.229.2:39553)
I20260811 22:22:31.688205 24340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:46127
WARNING: no policy specified for kudu/127.29.229.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
I20260811 22:22:31.740175 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:45475
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:22:32.108529 24824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:32.108909 24824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:32.109071 24824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 22:22:32.114693 24616 heartbeater.cc:503] Master 127.29.229.62:45475 was elected leader, sending a full tablet report...
W20260811 22:22:32.117192 24824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:32.117340 24824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:32.117515 24824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:22:32.127827 24824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:45475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:32.129181 24824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:32.130473 24824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:32.143998 24834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:32.143998 24829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:32.144035 24831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:32.144982 24824 server_base.cc:1034] running on GCE node
I20260811 22:22:32.145726 24824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:32.146853 24824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:32.148075 24824 hybrid_clock.cc:648] HybridClock initialized: now 1786486952148032 us; error 41 us; skew 500 ppm
I20260811 22:22:32.148367 24824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:32 dist-test-slave-jq4t krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486952, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:32.151687 24824 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:22:32.153338 24824 webserver.cc:460] Webserver started at http://127.29.229.3:40113/ using document root <none> and password file <none>
I20260811 22:22:32.154137 24824 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:32.154281 24824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:32.154613 24824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:22:32.157763 24824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "2376c62f68ec40f7ac5993e89fd86285"
format_stamp: "Formatted at 2026-08-11 22:22:32 on dist-test-slave-jq4t"
server_key: "3cdd2c34aac8a9b29fab5410d26ec1f3"
server_key_iv: "f9153a48dce6b669fc80a244da0788c3"
server_key_version: "encryptionkey@0"
I20260811 22:22:32.158573 24824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "2376c62f68ec40f7ac5993e89fd86285"
format_stamp: "Formatted at 2026-08-11 22:22:32 on dist-test-slave-jq4t"
server_key: "3cdd2c34aac8a9b29fab5410d26ec1f3"
server_key_iv: "f9153a48dce6b669fc80a244da0788c3"
server_key_version: "encryptionkey@0"
I20260811 22:22:32.164317 24824 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 22:22:32.167907 24846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:32.169301 24824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:22:32.169482 24824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "2376c62f68ec40f7ac5993e89fd86285"
format_stamp: "Formatted at 2026-08-11 22:22:32 on dist-test-slave-jq4t"
server_key: "3cdd2c34aac8a9b29fab5410d26ec1f3"
server_key_iv: "f9153a48dce6b669fc80a244da0788c3"
server_key_version: "encryptionkey@0"
I20260811 22:22:32.169698 24824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:32.185985 24824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:32.191196 24824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:32.191536 24824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:32.192649 24824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:32.194126 24824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:32.194205 24824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:32.194399 24824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:32.194455 24824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:32.231690 24824 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:39467
I20260811 22:22:32.231731 25017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:39467 every 8 connection(s)
I20260811 22:22:32.233072 24824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:22:32.240023 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 24824
I20260811 22:22:32.240259 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:22:32.240931 30612 external_mini_cluster.cc:1428] Setting key 16f7061e80e28398b5817e3af844ebd9
Aug 11 22:22:32 dist-test-slave-jq4t krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486952, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:22:32.255719 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:32.237101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:56753 (local address 127.29.229.62:45475)
0811 22:22:32.237641 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:32.237649 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:32.238119 (+ 470us) server_negotiation.cc:407] Connection header received
0811 22:22:32.240723 (+ 2604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:32.240729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:32.240839 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:32.240976 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:32.242727 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.243719 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.245502 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.245913 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.246010 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:32.249619 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:32.249624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:32.249627 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:32.249661 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:32.252456 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:32.253250 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:32.253254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:32.253256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:32.253375 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:32.253797 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:32.253801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:32.253803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:32.254110 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:32.254241 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:32.255078 (+ 837us) server_negotiation.cc:299] Negotiation successful
0811 22:22:32.255265 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":296,"threads_started":1}
I20260811 22:22:32.256816 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:32.236582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:56753)
0811 22:22:32.237917 (+ 1335us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:32.237968 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:32.240318 (+ 2350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:32.241265 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:32.241274 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:32.241793 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:32.242550 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:32.242568 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.243949 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.243954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:32.245362 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:32.245375 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.246151 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.246156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:32.246366 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:32.247018 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:32.247042 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:32.249394 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:32.252650 (+ 3256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:32.252657 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:32.252673 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:32.253129 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:32.253469 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:32.253472 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:32.253475 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:32.253654 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:32.254242 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:32.254247 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:32.254583 (+ 336us) client_negotiation.cc:769] Sending connection context
0811 22:22:32.254925 (+ 342us) client_negotiation.cc:240] Negotiation successful
0811 22:22:32.255477 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":979,"thread_start_us":776,"threads_started":1}
I20260811 22:22:32.258369 25018 heartbeater.cc:347] Connected to a master server at 127.29.229.62:45475
I20260811 22:22:32.258687 25018 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:32.259449 25018 heartbeater.cc:511] Master 127.29.229.62:45475 requested a full tablet report, sending...
I20260811 22:22:32.261760 24340 ts_manager.cc:194] Registered new tserver with Master: 2376c62f68ec40f7ac5993e89fd86285 (127.29.229.3:39467)
I20260811 22:22:32.262646 24340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:56753
I20260811 22:22:32.269124 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:22:32.287738 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:32.274075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47990 (local address 127.29.229.62:45475)
0811 22:22:32.274240 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:32.274244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:32.274800 (+ 556us) server_negotiation.cc:407] Connection header received
0811 22:22:32.275009 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:32.275020 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:32.275087 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:32.275193 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:32.276763 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.277575 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.279071 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.279496 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.279640 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:32.282301 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:32.282306 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:32.282310 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:32.282341 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:32.284927 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:32.285638 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:32.285648 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:32.285651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:32.285777 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:32.286187 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:32.286193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:32.286196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:32.286538 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:32.286814 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:32.287163 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 22:22:32.287414 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:22:32.292663 24340 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47990:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:22:32.299813 24340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:22:32.337965 25041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:32.314679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57516 (local address 127.29.229.2:39553)
0811 22:22:32.324615 (+ 9936us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:32.324629 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:32.326360 (+ 1731us) server_negotiation.cc:407] Connection header received
0811 22:22:32.326799 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:32.326811 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:32.327181 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:32.327436 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:32.329991 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.331478 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.334690 (+ 3212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.336280 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.336440 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:32.337039 (+ 599us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:32.337420 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 22:22:32.337616 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9712,"thread_start_us":9588,"threads_started":1}
I20260811 22:22:32.338902 25036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:32.313711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:39553 (local address 127.0.0.1:57516)
0811 22:22:32.326153 (+ 12442us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:32.326228 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:32.326502 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:32.329098 (+ 2596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:32.329110 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:32.329159 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:32.329759 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:32.329776 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.331650 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.331659 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:32.334473 (+ 2814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:32.334503 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.336782 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.336797 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:32.337095 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:32.337154 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 22:22:32.338285 (+ 1131us) client_negotiation.cc:240] Negotiation successful
0811 22:22:32.338521 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12075,"mutex_wait_us":45,"thread_start_us":13038,"threads_started":1}
I20260811 22:22:32.341441 25037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:32.314594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33947 (local address 127.0.0.1:35428)
0811 22:22:32.330820 (+ 16226us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:32.330846 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:32.331011 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:32.331922 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:32.331927 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:32.331955 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:32.332460 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:32.332492 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.334339 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.334346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:32.336702 (+ 2356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:32.336718 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.340519 (+ 3801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.340526 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:32.340718 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:32.340755 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 22:22:32.340847 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 22:22:32.341012 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16039,"mutex_wait_us":1,"thread_start_us":14149,"threads_started":1}
I20260811 22:22:32.342044 25040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:32.314197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:39467 (local address 127.0.0.1:57334)
0811 22:22:32.329054 (+ 14857us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:32.329074 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:32.329195 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:32.333454 (+ 4259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:32.333460 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:32.333486 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:32.333910 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:32.333921 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.335703 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.335710 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:32.338505 (+ 2795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:32.338520 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.340855 (+ 2335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.340862 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:32.341053 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:32.341085 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:22:32.341192 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 22:22:32.341343 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14724,"mutex_wait_us":3,"thread_start_us":16944,"threads_started":1}
I20260811 22:22:32.342413 25044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:32.315402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35428 (local address 127.29.229.1:33947)
0811 22:22:32.328032 (+ 12630us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:32.328044 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:32.331082 (+ 3038us) server_negotiation.cc:407] Connection header received
0811 22:22:32.331357 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:32.331370 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:32.331691 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:32.331931 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:32.332730 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.334199 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.338146 (+ 3947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.339613 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.339754 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:32.340331 (+ 577us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:32.341731 (+ 1400us) server_negotiation.cc:299] Negotiation successful
0811 22:22:32.342073 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12381,"thread_start_us":12183,"threads_started":1}
I20260811 22:22:32.343079 25039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:32.314329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57334 (local address 127.29.229.3:39467)
0811 22:22:32.332635 (+ 18306us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:32.332643 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:32.332684 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:22:32.332781 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:32.332789 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:32.333021 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:32.334119 (+ 1098us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:32.334402 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.335548 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.338684 (+ 3136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.340605 (+ 1921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.340738 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:32.341002 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:32.341254 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 22:22:32.341452 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18074,"thread_start_us":18838,"threads_started":1}
I20260811 22:22:32.343241 24720 tablet_service.cc:1467] Processing CreateTablet for tablet cf7f4ef84dbb432dab2a44bab4b6a630 (DEFAULT_TABLE table=test-table [id=63cd3bebb723474bafdf62f9a7bacc0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:22:32.344973 24720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf7f4ef84dbb432dab2a44bab4b6a630. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:32.346269 24526 tablet_service.cc:1467] Processing CreateTablet for tablet cf7f4ef84dbb432dab2a44bab4b6a630 (DEFAULT_TABLE table=test-table [id=63cd3bebb723474bafdf62f9a7bacc0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:22:32.347010 24918 tablet_service.cc:1467] Processing CreateTablet for tablet cf7f4ef84dbb432dab2a44bab4b6a630 (DEFAULT_TABLE table=test-table [id=63cd3bebb723474bafdf62f9a7bacc0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:22:32.347733 24526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf7f4ef84dbb432dab2a44bab4b6a630. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:32.348035 24918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf7f4ef84dbb432dab2a44bab4b6a630. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:32.359378 25045 tablet_bootstrap.cc:492] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285: Bootstrap starting.
I20260811 22:22:32.361598 25046 tablet_bootstrap.cc:492] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae: Bootstrap starting.
I20260811 22:22:32.363512 25045 tablet_bootstrap.cc:654] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:32.363656 25048 tablet_bootstrap.cc:492] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b: Bootstrap starting.
I20260811 22:22:32.365355 25046 tablet_bootstrap.cc:654] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:32.366832 25046 log.cc:824] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:32.366832 25045 log.cc:824] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:32.367172 25048 tablet_bootstrap.cc:654] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:32.368347 25048 log.cc:824] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:32.378093 25048 tablet_bootstrap.cc:492] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b: No bootstrap required, opened a new log
I20260811 22:22:32.378271 25045 tablet_bootstrap.cc:492] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285: No bootstrap required, opened a new log
I20260811 22:22:32.378412 25048 ts_tablet_manager.cc:1397] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260811 22:22:32.378609 25045 ts_tablet_manager.cc:1397] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.008s
I20260811 22:22:32.386760 25046 tablet_bootstrap.cc:492] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae: No bootstrap required, opened a new log
I20260811 22:22:32.386739 25045 raft_consensus.cc:348] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } }
I20260811 22:22:32.387216 25046 ts_tablet_manager.cc:1397] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae: Time spent bootstrapping tablet: real 0.026s user 0.010s sys 0.005s
I20260811 22:22:32.387403 25045 raft_consensus.cc:374] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:32.387638 25045 raft_consensus.cc:729] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2376c62f68ec40f7ac5993e89fd86285, State: Initialized, Role: FOLLOWER
I20260811 22:22:32.388626 25045 consensus_queue.cc:260] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } }
I20260811 22:22:32.389184 25048 raft_consensus.cc:348] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } }
I20260811 22:22:32.389663 25048 raft_consensus.cc:374] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:32.389797 25048 raft_consensus.cc:729] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 454fa69cf4d844b79d7688db9d8e722b, State: Initialized, Role: FOLLOWER
I20260811 22:22:32.390941 25018 heartbeater.cc:503] Master 127.29.229.62:45475 was elected leader, sending a full tablet report...
I20260811 22:22:32.391017 25048 consensus_queue.cc:260] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } }
I20260811 22:22:32.392902 25048 ts_tablet_manager.cc:1428] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260811 22:22:32.396248 25045 ts_tablet_manager.cc:1428] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285: Time spent starting tablet: real 0.018s user 0.010s sys 0.003s
I20260811 22:22:32.399624 25046 raft_consensus.cc:348] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } }
I20260811 22:22:32.400063 25046 raft_consensus.cc:374] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:32.400197 25046 raft_consensus.cc:729] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b9d9dfba06945f88970c30debaee5ae, State: Initialized, Role: FOLLOWER
I20260811 22:22:32.400852 25046 consensus_queue.cc:260] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } }
I20260811 22:22:32.401943 24815 heartbeater.cc:503] Master 127.29.229.62:45475 was elected leader, sending a full tablet report...
I20260811 22:22:32.402637 25046 ts_tablet_manager.cc:1428] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
W20260811 22:22:32.410764 24816 tablet.cc:2369] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:22:32.484865 25020 tablet.cc:2369] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:22:32.536125 25052 raft_consensus.cc:482] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:22:32.536545 25052 raft_consensus.cc:504] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } }
I20260811 22:22:32.538388 25052 leader_election.cc:288] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b9d9dfba06945f88970c30debaee5ae (127.29.229.2:39553), 454fa69cf4d844b79d7688db9d8e722b (127.29.229.1:33947)
I20260811 22:22:32.545432 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:32.538891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33947 (local address 127.29.229.3:55151)
0811 22:22:32.539163 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:32.539186 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:32.539328 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:32.539794 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:32.539799 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:32.539828 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:32.540077 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:32.540090 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.541401 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.541405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:32.543369 (+ 1964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:32.543384 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.544812 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.544816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:32.544966 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:32.544991 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:22:32.545067 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 22:22:32.545171 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 22:22:32.545532 25044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:32.538910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:55151 (local address 127.29.229.1:33947)
0811 22:22:32.539287 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:32.539295 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:32.539323 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:22:32.539511 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:32.539520 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:32.539620 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:32.539734 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:32.540227 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.541257 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.543539 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.544666 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.544832 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:32.544916 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:32.545128 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 22:22:32.545264 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214}
I20260811 22:22:32.546207 25055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:32.539077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:39553 (local address 127.29.229.3:58253)
0811 22:22:32.539895 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:32.539912 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:32.540037 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:32.540393 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:32.540396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:32.540415 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:32.540767 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:32.540774 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.541931 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.541936 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:32.543807 (+ 1871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:32.543817 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.545489 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.545498 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:32.545693 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:32.545727 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 22:22:32.545830 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 22:22:32.545942 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":485,"threads_started":1}
I20260811 22:22:32.546316 25041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:32.539037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:58253 (local address 127.29.229.2:39553)
0811 22:22:32.539313 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:32.539318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:32.540001 (+ 683us) server_negotiation.cc:407] Connection header received
0811 22:22:32.540182 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:32.540188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:32.540268 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:32.540373 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:32.540926 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.541772 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.543955 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.545263 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.545414 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:32.545483 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:32.545907 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 22:22:32.546048 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260811 22:22:32.546728 24556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf7f4ef84dbb432dab2a44bab4b6a630" candidate_uuid: "2376c62f68ec40f7ac5993e89fd86285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "454fa69cf4d844b79d7688db9d8e722b" is_pre_election: true
I20260811 22:22:32.547288 24556 raft_consensus.cc:2454] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2376c62f68ec40f7ac5993e89fd86285 in term 0.
I20260811 22:22:32.547318 24751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf7f4ef84dbb432dab2a44bab4b6a630" candidate_uuid: "2376c62f68ec40f7ac5993e89fd86285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b9d9dfba06945f88970c30debaee5ae" is_pre_election: true
I20260811 22:22:32.547840 24751 raft_consensus.cc:2454] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2376c62f68ec40f7ac5993e89fd86285 in term 0.
I20260811 22:22:32.548089 24852 leader_election.cc:302] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2376c62f68ec40f7ac5993e89fd86285, 454fa69cf4d844b79d7688db9d8e722b; no voters:
I20260811 22:22:32.548524 25052 raft_consensus.cc:2781] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:22:32.548700 25052 raft_consensus.cc:482] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:22:32.548832 25052 raft_consensus.cc:3037] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:32.551098 25052 raft_consensus.cc:504] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } }
I20260811 22:22:32.551709 25052 leader_election.cc:288] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [CANDIDATE]: Term 1 election: Requested vote from peers 6b9d9dfba06945f88970c30debaee5ae (127.29.229.2:39553), 454fa69cf4d844b79d7688db9d8e722b (127.29.229.1:33947)
I20260811 22:22:32.552237 24751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf7f4ef84dbb432dab2a44bab4b6a630" candidate_uuid: "2376c62f68ec40f7ac5993e89fd86285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b9d9dfba06945f88970c30debaee5ae"
I20260811 22:22:32.552238 24556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf7f4ef84dbb432dab2a44bab4b6a630" candidate_uuid: "2376c62f68ec40f7ac5993e89fd86285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "454fa69cf4d844b79d7688db9d8e722b"
I20260811 22:22:32.552462 24556 raft_consensus.cc:3037] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:32.552462 24751 raft_consensus.cc:3037] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:32.555775 24751 raft_consensus.cc:2454] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2376c62f68ec40f7ac5993e89fd86285 in term 1.
I20260811 22:22:32.555775 24556 raft_consensus.cc:2454] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2376c62f68ec40f7ac5993e89fd86285 in term 1.
I20260811 22:22:32.556203 24852 leader_election.cc:302] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2376c62f68ec40f7ac5993e89fd86285, 454fa69cf4d844b79d7688db9d8e722b; no voters:
I20260811 22:22:32.556596 25052 raft_consensus.cc:2781] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:22:32.557124 25052 raft_consensus.cc:686] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [term 1 LEADER]: Becoming Leader. State: Replica: 2376c62f68ec40f7ac5993e89fd86285, State: Running, Role: LEADER
I20260811 22:22:32.557745 25052 consensus_queue.cc:237] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } }
I20260811 22:22:32.562539 24340 catalog_manager.cc:5495] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2376c62f68ec40f7ac5993e89fd86285 (127.29.229.3). New cstate: current_term: 1 leader_uuid: "2376c62f68ec40f7ac5993e89fd86285" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } health_report { overall_health: UNKNOWN } } }
W20260811 22:22:32.578020 24619 tablet.cc:2369] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:22:32.634698 25039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:32.629407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57336 (local address 127.29.229.3:39467)
0811 22:22:32.629627 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:32.629632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:32.629652 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:22:32.629746 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:32.629749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:32.629831 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:32.629974 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:22:32.630421 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.631158 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.632681 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.633090 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.633187 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:32.633647 (+ 460us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:22:32.634149 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:22:32.634259 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:32.634416 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 22:22:32.634485 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 22:22:32.643014 24556 raft_consensus.cc:1261] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Refusing update from remote peer 2376c62f68ec40f7ac5993e89fd86285: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:22:32.643074 24751 raft_consensus.cc:1261] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae [term 1 FOLLOWER]: Refusing update from remote peer 2376c62f68ec40f7ac5993e89fd86285: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:22:32.644081 25057 consensus_queue.cc:1035] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:22:32.644515 25052 consensus_queue.cc:1035] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [LEADER]: Connected to new peer: Peer: permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:22:32.656886 25064 mvcc.cc:205] Tried to move back new op lower bound from 7317450558015901696 to 7317450557680713728. Current Snapshot: MvccSnapshot[applied={T|T < 7317450558015901696}]
I20260811 22:22:32.658847 25065 mvcc.cc:205] Tried to move back new op lower bound from 7317450558015901696 to 7317450557680713728. Current Snapshot: MvccSnapshot[applied={T|T < 7317450558015901696}]
I20260811 22:22:32.678841 24340 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:22:32.679150 24340 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:22:32.683610 24340 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6: Sending DeleteTablet for 3 replicas of tablet cf7f4ef84dbb432dab2a44bab4b6a630
I20260811 22:22:32.684973 24918 tablet_service.cc:1514] Processing DeleteTablet for tablet cf7f4ef84dbb432dab2a44bab4b6a630 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:22:32 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:57334
I20260811 22:22:32.684988 24720 tablet_service.cc:1514] Processing DeleteTablet for tablet cf7f4ef84dbb432dab2a44bab4b6a630 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:22:32 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:57516
I20260811 22:22:32.684999 24526 tablet_service.cc:1514] Processing DeleteTablet for tablet cf7f4ef84dbb432dab2a44bab4b6a630 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:22:32 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:35428
I20260811 22:22:32.689502 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 24437
I20260811 22:22:32.697490 25078 tablet_replica.cc:330] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285: stopping tablet replica
I20260811 22:22:32.698150 25078 raft_consensus.cc:2229] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:22:32.699172 25078 raft_consensus.cc:2258] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:22:32.701691 25078 ts_tablet_manager.cc:1905] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:22:32.710337 25078 ts_tablet_manager.cc:1918] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:22:32.710657 25078 log.cc:1196] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/cf7f4ef84dbb432dab2a44bab4b6a630
I20260811 22:22:32.711396 25078 ts_tablet_manager.cc:1939] T cf7f4ef84dbb432dab2a44bab4b6a630 P 2376c62f68ec40f7ac5993e89fd86285: Deleting consensus metadata
I20260811 22:22:32.713596 24319 catalog_manager.cc:4841] TS 2376c62f68ec40f7ac5993e89fd86285 (127.29.229.3:39467): tablet cf7f4ef84dbb432dab2a44bab4b6a630 (table test-table [id=63cd3bebb723474bafdf62f9a7bacc0e]) successfully deleted
I20260811 22:22:32.717119 25076 tablet_replica.cc:330] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b: stopping tablet replica
I20260811 22:22:32.717783 25076 raft_consensus.cc:2229] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:22:32.718470 25076 raft_consensus.cc:2258] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:22:32.719913 25077 tablet_replica.cc:330] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae: stopping tablet replica
I20260811 22:22:32.720396 25077 raft_consensus.cc:2229] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:22:32.720947 25076 ts_tablet_manager.cc:1905] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:22:32.720994 25077 raft_consensus.cc:2258] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:22:32.723219 25077 ts_tablet_manager.cc:1905] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 22:22:32 dist-test-slave-jq4t krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:22:32.728780 25076 ts_tablet_manager.cc:1918] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:22:32.729034 25076 log.cc:1196] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/wal/wals/cf7f4ef84dbb432dab2a44bab4b6a630
I20260811 22:22:32.729628 25076 ts_tablet_manager.cc:1939] T cf7f4ef84dbb432dab2a44bab4b6a630 P 454fa69cf4d844b79d7688db9d8e722b: Deleting consensus metadata
I20260811 22:22:32.730566 25077 ts_tablet_manager.cc:1918] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:22:32.730790 25077 log.cc:1196] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/cf7f4ef84dbb432dab2a44bab4b6a630
I20260811 22:22:32.731462 25077 ts_tablet_manager.cc:1939] T cf7f4ef84dbb432dab2a44bab4b6a630 P 6b9d9dfba06945f88970c30debaee5ae: Deleting consensus metadata
I20260811 22:22:32.731825 24323 catalog_manager.cc:4841] TS 454fa69cf4d844b79d7688db9d8e722b (127.29.229.1:33947): tablet cf7f4ef84dbb432dab2a44bab4b6a630 (table test-table [id=63cd3bebb723474bafdf62f9a7bacc0e]) successfully deleted
I20260811 22:22:32.733438 24321 catalog_manager.cc:4841] TS 6b9d9dfba06945f88970c30debaee5ae (127.29.229.2:39553): tablet cf7f4ef84dbb432dab2a44bab4b6a630 (table test-table [id=63cd3bebb723474bafdf62f9a7bacc0e]) successfully deleted
I20260811 22:22:32.734301 25044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:32.691366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35444 (local address 127.29.229.1:33947)
0811 22:22:32.691641 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:32.691651 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:32.709108 (+ 17457us) server_negotiation.cc:407] Connection header received
0811 22:22:32.709585 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:32.709594 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:32.709736 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:32.709854 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:32.713611 (+ 3757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.715185 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.718287 (+ 3102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.719236 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.719419 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:32.725525 (+ 6106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:32.725540 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:32.725557 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:32.725601 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:32.730168 (+ 4567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:32.731402 (+ 1234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:32.731410 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:32.731413 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:32.731737 (+ 324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:32.732225 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:32.732236 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:32.732239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:32.732728 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:32.733105 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:32.733777 (+ 672us) server_negotiation.cc:299] Negotiation successful
0811 22:22:32.734028 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260811 22:22:32.736949 24611 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:32.832837 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 24437
I20260811 22:22:32.867506 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 24625
Aug 11 22:22:32 dist-test-slave-jq4t krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.2@KRBTEST.COM
I20260811 22:22:32.886907 25041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:32.868606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57528 (local address 127.29.229.2:39553)
0811 22:22:32.868792 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:32.868796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:32.868816 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:22:32.868890 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:32.868893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:32.868957 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:32.869053 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:32.870975 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.871896 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.874094 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:32.874525 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:32.874724 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:32.880660 (+ 5936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:32.880679 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:32.880696 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:32.880735 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:32.883647 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:32.884623 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:32.884631 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:32.884633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:32.884769 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:32.885242 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:32.885246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:32.885248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:32.885649 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:32.885912 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:32.886309 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 22:22:32.886500 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:22:32.887768 24808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:32.981165 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 24625
I20260811 22:22:33.008248 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 24824
Aug 11 22:22:33 dist-test-slave-jq4t krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.3@KRBTEST.COM
I20260811 22:22:33.026268 25039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:33.009273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57340 (local address 127.29.229.3:39467)
0811 22:22:33.009494 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:33.009503 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:33.009529 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:22:33.009747 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:33.009752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:33.009822 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:33.009957 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:33.011328 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:33.012194 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:33.013757 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:33.014248 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:33.014412 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:33.019264 (+ 4852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:33.019278 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:33.019294 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:33.019335 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:33.022965 (+ 3630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:33.023821 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:33.023828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:33.023831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:33.024008 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:33.024473 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:33.024512 (+ 39us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:33.024518 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:33.024957 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:33.025311 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:33.025633 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 22:22:33.025866 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:22:33.027055 25011 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:33.156203 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 24824
I20260811 22:22:33.191766 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 24295
I20260811 22:22:33.193018 24386 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:33.307775 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 24295
I20260811 22:22:53.344033 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:45475
--webserver_interface=127.29.229.62
--webserver_port=41253
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:45475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:22:53.726011 25085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:53.726339 25085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:53.726511 25085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:53.734555 25085 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:22:53.734678 25085 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:22:53.734758 25085 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:22:53.734854 25085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:22:53.734951 25085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:53.735064 25085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:53.735126 25085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:22:53.735222 25085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:22:53.747884 25085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:45475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:45475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=41253
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:53.749388 25085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:53.750869 25085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:53.779812 25099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:53.779814 25106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:53.779855 25097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:53.780143 25085 server_base.cc:1034] running on GCE node
I20260811 22:22:53.781428 25085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:53.783200 25085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:53.784466 25085 hybrid_clock.cc:648] HybridClock initialized: now 1786486973784376 us; error 61 us; skew 500 ppm
I20260811 22:22:53.784934 25085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:53 dist-test-slave-jq4t krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486973, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:53.830482 25085 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:22:53.832677 25085 webserver.cc:460] Webserver started at http://127.29.229.62:41253/ using document root <none> and password file <none>
I20260811 22:22:53.833528 25085 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:53.833647 25085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:53.840126 25085 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:22:53.843762 25115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:53.845431 25085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:22:53.845613 25085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6"
format_stamp: "Formatted at 2026-08-11 22:22:30 on dist-test-slave-jq4t"
server_key: "b7aa420b88eac14b438c66c7ce86e61b"
server_key_iv: "90b12d4a88f80c6e71b51e48f4dab6db"
server_key_version: "encryptionkey@0"
I20260811 22:22:53.846750 25085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:53.874619 25085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:53.879272 25085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:53.879539 25085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:53.903055 25085 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:45475
I20260811 22:22:53.903105 25190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:45475 every 8 connection(s)
I20260811 22:22:53.904762 25085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:22:53.905750 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 25085
I20260811 22:22:53.906806 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:33947
--local_ip_for_outbound_sockets=127.29.229.1
--tserver_master_addrs=127.29.229.62:45475
--webserver_port=38211
--webserver_interface=127.29.229.1
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:22:53.912657 25191 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:22:53.916235 25191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6: Bootstrap starting.
I20260811 22:22:53.942855 25191 log.cc:824] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:53.958541 25191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:22:53.959160 25191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6: Bootstrap complete.
I20260811 22:22:53.968766 25191 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45475 } }
I20260811 22:22:53.969632 25191 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a3ecbba44a646a1bba5bb4f0fa56ba6, State: Initialized, Role: FOLLOWER
I20260811 22:22:53.970265 25191 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45475 } }
I20260811 22:22:53.970599 25191 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:22:53.970765 25191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:22:53.970971 25191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:22:53.975517 25191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45475 } }
I20260811 22:22:53.976071 25191 leader_election.cc:302] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a3ecbba44a646a1bba5bb4f0fa56ba6; no voters:
I20260811 22:22:53.976701 25191 leader_election.cc:288] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:22:53.976908 25199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:22:53.978209 25199 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 2 LEADER]: Becoming Leader. State: Replica: 6a3ecbba44a646a1bba5bb4f0fa56ba6, State: Running, Role: LEADER
I20260811 22:22:53.979009 25191 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:22:53.978895 25199 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45475 } }
I20260811 22:22:53.982074 25201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a3ecbba44a646a1bba5bb4f0fa56ba6. Latest consensus state: current_term: 2 leader_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45475 } } }
I20260811 22:22:53.982568 25201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:22:53.983196 25200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45475 } } }
I20260811 22:22:53.983618 25200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:22:53.993777 25212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:22:54.004344 25212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:22:54.005918 25212 catalog_manager.cc:1259] Loaded cluster ID: a5362d30f0074d67a133a754b4584d98
I20260811 22:22:54.006028 25212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:22:54.010210 25212 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:22:54.012063 25212 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6: Loaded TSK: 0
I20260811 22:22:54.058916 25212 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6: Generated new TSK 1
I20260811 22:22:54.059351 25212 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:22:54.336905 25195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:54.337185 25195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:54.337337 25195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:54.345091 25195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:54.345221 25195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:54.345391 25195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:22:54.356339 25195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:33947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=38211
--tserver_master_addrs=127.29.229.62:45475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:54.357769 25195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:54.359211 25195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:54.374547 25239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:54.374662 25241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:54.375358 25238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:54.376080 25195 server_base.cc:1034] running on GCE node
I20260811 22:22:54.376782 25195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:54.377875 25195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:54.379151 25195 hybrid_clock.cc:648] HybridClock initialized: now 1786486974379072 us; error 36 us; skew 500 ppm
I20260811 22:22:54.379549 25195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:54 dist-test-slave-jq4t krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486974, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:54.383091 25195 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:22:54.384781 25195 webserver.cc:460] Webserver started at http://127.29.229.1:38211/ using document root <none> and password file <none>
I20260811 22:22:54.385521 25195 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:54.385689 25195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:54.391425 25195 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260811 22:22:54.394572 25251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:54.396095 25195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 22:22:54.396301 25195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "454fa69cf4d844b79d7688db9d8e722b"
format_stamp: "Formatted at 2026-08-11 22:22:30 on dist-test-slave-jq4t"
server_key: "e0f3998d51d997339b594aaa503b1e17"
server_key_iv: "6f9bef3f96de25cd0a983527762693dd"
server_key_version: "encryptionkey@0"
I20260811 22:22:54.397433 25195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:54.411531 25195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:54.415813 25195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:54.416108 25195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:54.417215 25195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:54.418704 25195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:54.418819 25195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:54.418944 25195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:54.419039 25195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:54.455415 25195 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:33947
I20260811 22:22:54.455506 25440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:33947 every 8 connection(s)
I20260811 22:22:54.457139 25195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:22:54.461982 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 25195
I20260811 22:22:54.463160 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:39553
--local_ip_for_outbound_sockets=127.29.229.2
--tserver_master_addrs=127.29.229.62:45475
--webserver_port=44573
--webserver_interface=127.29.229.2
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:22:54 dist-test-slave-jq4t krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486974, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:22:54.494086 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:54.460693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:40313)
0811 22:22:54.461783 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:54.461845 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:54.465464 (+ 3619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:54.467629 (+ 2165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:54.467642 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:54.468558 (+ 916us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:54.469379 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:54.469409 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:54.474586 (+ 5177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:54.474595 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:54.476958 (+ 2363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:54.476979 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:54.478445 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:54.478455 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:54.478792 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:54.480211 (+ 1419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:54.480263 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:54.483397 (+ 3134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:54.489169 (+ 5772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:54.489183 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:54.489202 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:54.489734 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:54.490331 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:54.490335 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:54.490338 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:54.490550 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:54.491687 (+ 1137us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:54.491693 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:54.492115 (+ 422us) client_negotiation.cc:769] Sending connection context
0811 22:22:54.492374 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 22:22:54.492884 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":385,"threads_started":1}
I20260811 22:22:54.494860 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:54.460162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:40313 (local address 127.29.229.62:45475)
0811 22:22:54.460786 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:54.460798 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:54.462042 (+ 1244us) server_negotiation.cc:407] Connection header received
0811 22:22:54.466351 (+ 4309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:54.466371 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:54.467000 (+ 629us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:54.467366 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:54.469662 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:54.474341 (+ 4679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:54.477137 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:54.478159 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:54.478359 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:54.483791 (+ 5432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:54.483806 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:54.483829 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:54.483888 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:54.488946 (+ 5058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:54.489926 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:54.489940 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:54.489951 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:54.490187 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:54.490684 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:54.490692 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:54.490698 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:54.491494 (+ 796us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:54.491732 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:54.492708 (+ 976us) server_negotiation.cc:299] Negotiation successful
0811 22:22:54.493208 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":299,"threads_started":1}
I20260811 22:22:54.497709 25442 heartbeater.cc:347] Connected to a master server at 127.29.229.62:45475
I20260811 22:22:54.498091 25442 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:54.499480 25442 heartbeater.cc:511] Master 127.29.229.62:45475 requested a full tablet report, sending...
I20260811 22:22:54.501889 25145 ts_manager.cc:194] Registered new tserver with Master: 454fa69cf4d844b79d7688db9d8e722b (127.29.229.1:33947)
I20260811 22:22:54.503430 25145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:40313
W20260811 22:22:54.847426 25446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:54.847786 25446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:54.847968 25446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:54.855589 25446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:54.855756 25446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:54.855945 25446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:22:54.865962 25446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:39553
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=44573
--tserver_master_addrs=127.29.229.62:45475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:54.867416 25446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:54.869123 25446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:54.883786 25451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:54.884282 25452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:54.884300 25455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:54.884825 25446 server_base.cc:1034] running on GCE node
I20260811 22:22:54.885563 25446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:54.886752 25446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:54.888048 25446 hybrid_clock.cc:648] HybridClock initialized: now 1786486974887983 us; error 71 us; skew 500 ppm
I20260811 22:22:54.888437 25446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:54 dist-test-slave-jq4t krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486974, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:54.892091 25446 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:22:54.893863 25446 webserver.cc:460] Webserver started at http://127.29.229.2:44573/ using document root <none> and password file <none>
I20260811 22:22:54.894613 25446 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:54.894778 25446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:54.901135 25446 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 22:22:54.905918 25468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:54.907784 25446 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260811 22:22:54.907992 25446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "6b9d9dfba06945f88970c30debaee5ae"
format_stamp: "Formatted at 2026-08-11 22:22:31 on dist-test-slave-jq4t"
server_key: "0b4fa2daba7d77bed1e8c60374d378d9"
server_key_iv: "59379c81bfd6c61a184b79d6ee436e13"
server_key_version: "encryptionkey@0"
I20260811 22:22:54.909554 25446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:54.929091 25446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:54.933360 25446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:54.933614 25446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:54.934810 25446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:54.936381 25446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:54.936563 25446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:54.936666 25446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:54.936731 25446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:54.975646 25446 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:39553
I20260811 22:22:54.975699 25641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:39553 every 8 connection(s)
I20260811 22:22:54.977183 25446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:22:54.977712 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 25446
I20260811 22:22:54.978569 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:39467
--local_ip_for_outbound_sockets=127.29.229.3
--tserver_master_addrs=127.29.229.62:45475
--webserver_port=40113
--webserver_interface=127.29.229.3
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:22:54 dist-test-slave-jq4t krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486974, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:22:55.005280 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:54.980510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:39763 (local address 127.29.229.62:45475)
0811 22:22:54.980750 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:54.980759 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:54.982091 (+ 1332us) server_negotiation.cc:407] Connection header received
0811 22:22:54.985138 (+ 3047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:54.985143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:54.985241 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:54.985341 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:54.987738 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:54.988764 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:54.991365 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:54.991866 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:54.991987 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:54.996345 (+ 4358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:54.996355 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:54.996363 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:54.996424 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:55.000448 (+ 4024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:55.001444 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:55.001453 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:55.001460 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:55.001637 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:55.002393 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:55.002403 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:55.002409 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:55.003157 (+ 748us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:55.003336 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:55.004305 (+ 969us) server_negotiation.cc:299] Negotiation successful
0811 22:22:55.004639 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:22:55.005822 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:54.981013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:39763)
0811 22:22:54.981883 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:54.981942 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:54.984760 (+ 2818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:54.985652 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:54.985667 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:54.986549 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:54.987488 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:54.987521 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:54.988943 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:54.988951 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:54.991182 (+ 2231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:54.991203 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:54.992130 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:54.992139 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:54.992528 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:54.993415 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:54.993452 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:54.996047 (+ 2595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:55.000682 (+ 4635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:55.000690 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:55.000704 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:55.001285 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:55.001793 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:55.001797 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:55.001799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:55.002038 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:55.003324 (+ 1286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:55.003331 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:55.003819 (+ 488us) client_negotiation.cc:769] Sending connection context
0811 22:22:55.004096 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 22:22:55.004618 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":317,"threads_started":1}
I20260811 22:22:55.007577 25645 heartbeater.cc:347] Connected to a master server at 127.29.229.62:45475
I20260811 22:22:55.007927 25645 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:55.008788 25645 heartbeater.cc:511] Master 127.29.229.62:45475 requested a full tablet report, sending...
I20260811 22:22:55.010461 25145 ts_manager.cc:194] Registered new tserver with Master: 6b9d9dfba06945f88970c30debaee5ae (127.29.229.2:39553)
I20260811 22:22:55.011479 25145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:39763
W20260811 22:22:55.359062 25648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:22:55.359360 25648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:22:55.359519 25648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:22:55.366971 25648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:22:55.367092 25648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:22:55.367283 25648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:22:55.377717 25648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:39467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=40113
--tserver_master_addrs=127.29.229.62:45475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:22:55.379114 25648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:22:55.381131 25648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:22:55.397738 25657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:55.397831 25659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:22:55.398025 25662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:22:55.398623 25648 server_base.cc:1034] running on GCE node
I20260811 22:22:55.399297 25648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:22:55.400424 25648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:22:55.401702 25648 hybrid_clock.cc:648] HybridClock initialized: now 1786486975401625 us; error 63 us; skew 500 ppm
I20260811 22:22:55.402041 25648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:22:55 dist-test-slave-jq4t krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486975, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:22:55.405882 25648 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:22:55.407508 25648 webserver.cc:460] Webserver started at http://127.29.229.3:40113/ using document root <none> and password file <none>
I20260811 22:22:55.408282 25648 fs_manager.cc:359] Metadata directory not provided
I20260811 22:22:55.408448 25648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:22:55.414892 25648 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:22:55.417934 25670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:55.419348 25648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:22:55.419539 25648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "2376c62f68ec40f7ac5993e89fd86285"
format_stamp: "Formatted at 2026-08-11 22:22:32 on dist-test-slave-jq4t"
server_key: "3cdd2c34aac8a9b29fab5410d26ec1f3"
server_key_iv: "f9153a48dce6b669fc80a244da0788c3"
server_key_version: "encryptionkey@0"
I20260811 22:22:55.420596 25648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:22:55.433239 25648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:22:55.437547 25648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:22:55.437879 25648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:22:55.438937 25648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:22:55.440339 25648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:22:55.440464 25648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:55.440608 25648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:22:55.440703 25648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:22:55.479712 25648 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:39467
I20260811 22:22:55.479764 25843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:39467 every 8 connection(s)
I20260811 22:22:55.481406 25648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:22:55.482100 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 25648
Aug 11 22:22:55 dist-test-slave-jq4t krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486975, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:22:55.498895 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:55.483328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33932 (local address 127.29.229.62:45475)
0811 22:22:55.483574 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:55.483580 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:55.484034 (+ 454us) server_negotiation.cc:407] Connection header received
0811 22:22:55.484288 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:55.484293 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:55.484387 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:55.484538 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:55.486886 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.487946 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.489377 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.489863 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.490010 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:55.492882 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:55.492888 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:55.492891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:55.492921 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:55.495865 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:55.496778 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:55.496782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:55.496785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:55.496913 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:55.497326 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:55.497331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:55.497335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:55.497610 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:55.497891 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:55.498245 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 22:22:55.498495 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 22:22:55.504033 25849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:55.484783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:52853 (local address 127.29.229.62:45475)
0811 22:22:55.485620 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:55.485641 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:55.487388 (+ 1747us) server_negotiation.cc:407] Connection header received
0811 22:22:55.489310 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:55.489315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:55.489403 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:55.489533 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:55.491040 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.491777 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.493391 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.494025 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.494182 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:55.497858 (+ 3676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:55.497869 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:55.497874 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:55.497904 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:55.500889 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:55.501674 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:55.501678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:55.501681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:55.501799 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:55.502242 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:55.502247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:55.502250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:55.502518 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:55.502690 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:55.503395 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 22:22:55.503634 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":264,"threads_started":1}
I20260811 22:22:55.505033 25850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:55.485077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:52853)
0811 22:22:55.487233 (+ 2156us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:55.487278 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:55.489066 (+ 1788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:55.489778 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:55.489787 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:55.490289 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:22:55.490873 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:55.490890 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.491937 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.491942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:55.493234 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:55.493247 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.494264 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.494273 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:55.494544 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:55.495187 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:22:55.495209 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:22:55.497631 (+ 2422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:22:55.501041 (+ 3410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:55.501055 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:55.501068 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:55.501537 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:55.501917 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:22:55.501922 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:22:55.501926 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:22:55.502119 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:22:55.502648 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:22:55.502657 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:22:55.503016 (+ 359us) client_negotiation.cc:769] Sending connection context
0811 22:22:55.503277 (+ 261us) client_negotiation.cc:240] Negotiation successful
0811 22:22:55.503830 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1907,"thread_start_us":1724,"threads_started":1}
I20260811 22:22:55.506397 25844 heartbeater.cc:347] Connected to a master server at 127.29.229.62:45475
I20260811 22:22:55.506757 25844 heartbeater.cc:464] Registering TS with master...
I20260811 22:22:55.507434 25844 heartbeater.cc:511] Master 127.29.229.62:45475 requested a full tablet report, sending...
I20260811 22:22:55.507786 25442 heartbeater.cc:503] Master 127.29.229.62:45475 was elected leader, sending a full tablet report...
I20260811 22:22:55.509182 25143 ts_manager.cc:194] Registered new tserver with Master: 2376c62f68ec40f7ac5993e89fd86285 (127.29.229.3:39467)
I20260811 22:22:55.510154 25143 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:52853
I20260811 22:22:55.514096 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:22:55.520983 25849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:55.515125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33934 (local address 127.29.229.62:45475)
0811 22:22:55.515340 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:55.515347 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:55.515734 (+ 387us) server_negotiation.cc:407] Connection header received
0811 22:22:55.515957 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:55.515960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:55.516098 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:55.516223 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:22:55.516717 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.517449 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.518938 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.519407 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.519508 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:55.520026 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:22:55.520371 (+ 345us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 22:22:55.520750 (+ 379us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33934: token expired
Metrics: {"server-negotiator.queue_time_us":74}
W20260811 22:22:55.521090 25849 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33934: token expired
W20260811 22:22:55.522109 25025 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.229.62:45475)
I20260811 22:22:55.535683 25849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:55.522704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33940 (local address 127.29.229.62:45475)
0811 22:22:55.522850 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:55.522854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:55.522870 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:22:55.522991 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:55.522994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:55.523052 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:55.523133 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:55.524659 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.525503 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.526998 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.527459 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.527565 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:55.530413 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:55.530418 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:55.530422 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:55.530457 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:55.533145 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:55.533985 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:55.533989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:55.533991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:55.534114 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:55.534494 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:55.534497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:55.534499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:55.534802 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:55.535033 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:55.535268 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 22:22:55.535441 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:22:55.539386 25143 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:22:55.548403 25143 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:22:55.587374 25860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:55.561279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50110 (local address 127.29.229.1:33947)
0811 22:22:55.577748 (+ 16469us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:55.577762 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:55.577801 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:22:55.577944 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:55.577956 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:55.578282 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:55.580511 (+ 2229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:55.580781 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.582466 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.584969 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.586101 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.586264 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:55.586526 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:55.586864 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 22:22:55.587035 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16219,"thread_start_us":16067,"threads_started":1}
I20260811 22:22:55.587426 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:55.561030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33947 (local address 127.0.0.1:50110)
0811 22:22:55.573805 (+ 12775us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:55.573853 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:55.574041 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:55.579640 (+ 5599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:55.579652 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:55.579697 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:55.580193 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:55.580205 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.582634 (+ 2429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.582642 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:55.584817 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:55.584831 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.586350 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.586357 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:55.586579 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:55.586635 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 22:22:55.586773 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 22:22:55.586985 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12518,"mutex_wait_us":169,"thread_start_us":12338,"threads_started":1}
I20260811 22:22:55.588755 25862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:55.561670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:39553 (local address 127.0.0.1:39830)
0811 22:22:55.578867 (+ 17197us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:55.578883 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:55.578981 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:55.580663 (+ 1682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:55.580673 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:55.580701 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:55.581080 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:55.581089 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.582780 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.582785 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:55.585566 (+ 2781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:55.585583 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.587885 (+ 2302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.587892 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:55.588089 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:55.588135 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 22:22:55.588231 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 22:22:55.588361 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17066,"mutex_wait_us":118,"thread_start_us":16177,"threads_started":1}
I20260811 22:22:55.589300 25864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:55.561931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39830 (local address 127.29.229.2:39553)
0811 22:22:55.580061 (+ 18130us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:55.580072 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:55.580114 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:22:55.580209 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:55.580215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:55.580426 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:55.580670 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:55.581247 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.582610 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.585775 (+ 3165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.587605 (+ 1830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.587757 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:55.587951 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:55.588621 (+ 670us) server_negotiation.cc:299] Negotiation successful
0811 22:22:55.589003 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17968,"thread_start_us":17860,"threads_started":1}
I20260811 22:22:55.589701 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:55.561143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:39467 (local address 127.0.0.1:53786)
0811 22:22:55.578107 (+ 16964us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:55.578133 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:55.578285 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:55.580193 (+ 1908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:55.580197 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:55.580223 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:55.580624 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:55.580638 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.582988 (+ 2350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.583000 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:55.586455 (+ 3455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:55.586469 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.588841 (+ 2372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.588847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:55.589056 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:55.589090 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 22:22:55.589181 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:22:55.589332 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16830,"mutex_wait_us":89,"thread_start_us":17434,"threads_started":1}
I20260811 22:22:55.590583 25865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:55.561846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53786 (local address 127.29.229.3:39467)
0811 22:22:55.579601 (+ 17755us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:55.579611 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:55.579645 (+ 34us) server_negotiation.cc:407] Connection header received
0811 22:22:55.579750 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:55.579759 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:55.579980 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:55.580188 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:55.580851 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.582861 (+ 2010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.586768 (+ 3907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.588303 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.588436 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:55.588668 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:55.590072 (+ 1404us) server_negotiation.cc:299] Negotiation successful
0811 22:22:55.590197 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17564,"thread_start_us":17424,"threads_started":1}
I20260811 22:22:55.592597 25542 tablet_service.cc:1467] Processing CreateTablet for tablet 35d0e80468d74c55be594d752d307060 (DEFAULT_TABLE table=test-table [id=e7ed4aed59684ca1956b76693db382ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:22:55.592442 25331 tablet_service.cc:1467] Processing CreateTablet for tablet 35d0e80468d74c55be594d752d307060 (DEFAULT_TABLE table=test-table [id=e7ed4aed59684ca1956b76693db382ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:22:55.593735 25542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35d0e80468d74c55be594d752d307060. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:55.593959 25331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35d0e80468d74c55be594d752d307060. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:55.593982 25741 tablet_service.cc:1467] Processing CreateTablet for tablet 35d0e80468d74c55be594d752d307060 (DEFAULT_TABLE table=test-table [id=e7ed4aed59684ca1956b76693db382ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:22:55.595311 25741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35d0e80468d74c55be594d752d307060. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:22:55.606992 25866 tablet_bootstrap.cc:492] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae: Bootstrap starting.
I20260811 22:22:55.610841 25870 tablet_bootstrap.cc:492] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285: Bootstrap starting.
I20260811 22:22:55.611714 25866 tablet_bootstrap.cc:654] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:55.612562 25868 tablet_bootstrap.cc:492] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b: Bootstrap starting.
I20260811 22:22:55.613305 25866 log.cc:824] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:55.614655 25870 tablet_bootstrap.cc:654] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:55.616169 25868 tablet_bootstrap.cc:654] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b: Neither blocks nor log segments found. Creating new log.
I20260811 22:22:55.617000 25870 log.cc:824] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:55.617172 25866 tablet_bootstrap.cc:492] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae: No bootstrap required, opened a new log
I20260811 22:22:55.617476 25866 ts_tablet_manager.cc:1397] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 22:22:55.617498 25868 log.cc:824] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:22:55.620180 25870 tablet_bootstrap.cc:492] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285: No bootstrap required, opened a new log
I20260811 22:22:55.620497 25870 ts_tablet_manager.cc:1397] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260811 22:22:55.620633 25868 tablet_bootstrap.cc:492] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b: No bootstrap required, opened a new log
I20260811 22:22:55.620922 25868 ts_tablet_manager.cc:1397] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 22:22:55.625722 25866 raft_consensus.cc:348] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } }
I20260811 22:22:55.626231 25866 raft_consensus.cc:374] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:55.626343 25866 raft_consensus.cc:729] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b9d9dfba06945f88970c30debaee5ae, State: Initialized, Role: FOLLOWER
I20260811 22:22:55.626987 25866 consensus_queue.cc:260] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } }
I20260811 22:22:55.628180 25645 heartbeater.cc:503] Master 127.29.229.62:45475 was elected leader, sending a full tablet report...
I20260811 22:22:55.628957 25866 ts_tablet_manager.cc:1428] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260811 22:22:55.633534 25870 raft_consensus.cc:348] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } }
I20260811 22:22:55.634053 25870 raft_consensus.cc:374] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:55.634151 25870 raft_consensus.cc:729] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2376c62f68ec40f7ac5993e89fd86285, State: Initialized, Role: FOLLOWER
I20260811 22:22:55.634805 25870 consensus_queue.cc:260] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } }
I20260811 22:22:55.635095 25868 raft_consensus.cc:348] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } }
I20260811 22:22:55.635596 25868 raft_consensus.cc:374] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:22:55.635694 25868 raft_consensus.cc:729] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 454fa69cf4d844b79d7688db9d8e722b, State: Initialized, Role: FOLLOWER
I20260811 22:22:55.636420 25844 heartbeater.cc:503] Master 127.29.229.62:45475 was elected leader, sending a full tablet report...
I20260811 22:22:55.636291 25868 consensus_queue.cc:260] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } }
I20260811 22:22:55.637008 25870 ts_tablet_manager.cc:1428] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285: Time spent starting tablet: real 0.016s user 0.015s sys 0.001s
I20260811 22:22:55.638013 25868 ts_tablet_manager.cc:1428] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
W20260811 22:22:55.709689 25443 tablet.cc:2369] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:22:55.729534 25646 tablet.cc:2369] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:22:55.731565 25876 raft_consensus.cc:482] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:22:55.731946 25876 raft_consensus.cc:504] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } }
W20260811 22:22:55.733639 25846 tablet.cc:2369] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:22:55.734025 25876 leader_election.cc:288] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 454fa69cf4d844b79d7688db9d8e722b (127.29.229.1:33947), 2376c62f68ec40f7ac5993e89fd86285 (127.29.229.3:39467)
I20260811 22:22:55.743227 25860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:55.734884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:49623 (local address 127.29.229.1:33947)
0811 22:22:55.735292 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:55.735301 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:55.736071 (+ 770us) server_negotiation.cc:407] Connection header received
0811 22:22:55.736301 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:55.736305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:55.736367 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:55.736465 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:55.737108 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.737958 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.740695 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.741989 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.742175 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:55.742243 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:55.742818 (+ 575us) server_negotiation.cc:299] Negotiation successful
0811 22:22:55.742960 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210}
I20260811 22:22:55.743222 25865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:55.734683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:40507 (local address 127.29.229.3:39467)
0811 22:22:55.734948 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:55.734957 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:55.735770 (+ 813us) server_negotiation.cc:407] Connection header received
0811 22:22:55.736009 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:55.736013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:55.736074 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:55.736174 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:22:55.736803 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.737752 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.740690 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.741990 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.742166 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:55.742243 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:55.742806 (+ 563us) server_negotiation.cc:299] Negotiation successful
0811 22:22:55.742956 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 22:22:55.743446 25881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:55.734439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33947 (local address 127.29.229.2:49623)
0811 22:22:55.735929 (+ 1490us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:55.735945 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:55.736149 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:55.736512 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:55.736516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:55.736546 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:55.736948 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:55.736957 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.738146 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.738153 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:55.740493 (+ 2340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:55.740509 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.742317 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.742324 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:55.742572 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:55.742655 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 22:22:55.742755 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 22:22:55.742955 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1327,"mutex_wait_us":151,"thread_start_us":1107,"threads_started":1}
I20260811 22:22:55.743855 25880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:55.734321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:39467 (local address 127.29.229.2:40507)
0811 22:22:55.735627 (+ 1306us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:55.735651 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:22:55.735845 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:22:55.736191 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:22:55.736196 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:22:55.736223 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:22:55.736653 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:55.736664 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.737927 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.737933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:55.740500 (+ 2567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:22:55.740509 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.742321 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.742326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:22:55.742567 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:55.742653 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 22:22:55.742755 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:22:55.742955 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1171,"mutex_wait_us":116,"thread_start_us":938,"threads_started":1}
I20260811 22:22:55.744619 25361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35d0e80468d74c55be594d752d307060" candidate_uuid: "6b9d9dfba06945f88970c30debaee5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "454fa69cf4d844b79d7688db9d8e722b" is_pre_election: true
I20260811 22:22:55.745215 25776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35d0e80468d74c55be594d752d307060" candidate_uuid: "6b9d9dfba06945f88970c30debaee5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2376c62f68ec40f7ac5993e89fd86285" is_pre_election: true
I20260811 22:22:55.745414 25361 raft_consensus.cc:2454] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b9d9dfba06945f88970c30debaee5ae in term 0.
I20260811 22:22:55.745730 25776 raft_consensus.cc:2454] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b9d9dfba06945f88970c30debaee5ae in term 0.
I20260811 22:22:55.746201 25474 leader_election.cc:302] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 454fa69cf4d844b79d7688db9d8e722b, 6b9d9dfba06945f88970c30debaee5ae; no voters:
I20260811 22:22:55.746646 25876 raft_consensus.cc:2781] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:22:55.746793 25876 raft_consensus.cc:482] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:22:55.746991 25876 raft_consensus.cc:3037] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:55.749615 25876 raft_consensus.cc:504] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } }
I20260811 22:22:55.750310 25876 leader_election.cc:288] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [CANDIDATE]: Term 1 election: Requested vote from peers 454fa69cf4d844b79d7688db9d8e722b (127.29.229.1:33947), 2376c62f68ec40f7ac5993e89fd86285 (127.29.229.3:39467)
I20260811 22:22:55.750896 25776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35d0e80468d74c55be594d752d307060" candidate_uuid: "6b9d9dfba06945f88970c30debaee5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2376c62f68ec40f7ac5993e89fd86285"
I20260811 22:22:55.751144 25776 raft_consensus.cc:3037] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:55.751108 25361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35d0e80468d74c55be594d752d307060" candidate_uuid: "6b9d9dfba06945f88970c30debaee5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "454fa69cf4d844b79d7688db9d8e722b"
I20260811 22:22:55.751343 25361 raft_consensus.cc:3037] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:22:55.753611 25776 raft_consensus.cc:2454] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b9d9dfba06945f88970c30debaee5ae in term 1.
I20260811 22:22:55.754056 25361 raft_consensus.cc:2454] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b9d9dfba06945f88970c30debaee5ae in term 1.
I20260811 22:22:55.754240 25470 leader_election.cc:302] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2376c62f68ec40f7ac5993e89fd86285, 6b9d9dfba06945f88970c30debaee5ae; no voters:
I20260811 22:22:55.754633 25876 raft_consensus.cc:2781] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:22:55.755208 25876 raft_consensus.cc:686] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [term 1 LEADER]: Becoming Leader. State: Replica: 6b9d9dfba06945f88970c30debaee5ae, State: Running, Role: LEADER
I20260811 22:22:55.755968 25876 consensus_queue.cc:237] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } }
I20260811 22:22:55.760885 25143 catalog_manager.cc:5495] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b9d9dfba06945f88970c30debaee5ae (127.29.229.2). New cstate: current_term: 1 leader_uuid: "6b9d9dfba06945f88970c30debaee5ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } health_report { overall_health: HEALTHY } } }
I20260811 22:22:55.794042 25864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:55.783662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39840 (local address 127.29.229.2:39553)
0811 22:22:55.783900 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:55.783908 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:55.783933 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:22:55.784061 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:55.784066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:55.784200 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:55.784375 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:22:55.784960 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.786179 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.788643 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.789425 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.789602 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:55.792693 (+ 3091us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:22:55.793247 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:22:55.793467 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:55.793559 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 22:22:55.793675 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:22:55.803321 25361 raft_consensus.cc:1261] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Refusing update from remote peer 6b9d9dfba06945f88970c30debaee5ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:22:55.803328 25776 raft_consensus.cc:1261] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285 [term 1 FOLLOWER]: Refusing update from remote peer 6b9d9dfba06945f88970c30debaee5ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:22:55.804452 25882 consensus_queue.cc:1035] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:22:55.804867 25876 consensus_queue.cc:1035] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:22:55.816882 25887 mvcc.cc:205] Tried to move back new op lower bound from 7317450652878835712 to 7317450652700676096. Current Snapshot: MvccSnapshot[applied={T|T < 7317450652878835712}]
I20260811 22:22:55.818302 25886 mvcc.cc:205] Tried to move back new op lower bound from 7317450652878835712 to 7317450652700676096. Current Snapshot: MvccSnapshot[applied={T|T < 7317450652878835712}]
I20260811 22:22:55.833832 25143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:22:55.834086 25143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:22:55.837477 25143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6: Sending DeleteTablet for 3 replicas of tablet 35d0e80468d74c55be594d752d307060
I20260811 22:22:55.838726 25331 tablet_service.cc:1514] Processing DeleteTablet for tablet 35d0e80468d74c55be594d752d307060 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:22:55 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:50110
I20260811 22:22:55.838732 25542 tablet_service.cc:1514] Processing DeleteTablet for tablet 35d0e80468d74c55be594d752d307060 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:22:55 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:39830
I20260811 22:22:55.838718 25741 tablet_service.cc:1514] Processing DeleteTablet for tablet 35d0e80468d74c55be594d752d307060 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:22:55 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:53786
I20260811 22:22:55.839471 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 25195
I20260811 22:22:55.851140 25900 tablet_replica.cc:330] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b: stopping tablet replica
I20260811 22:22:55.851688 25900 raft_consensus.cc:2229] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:22:55.852275 25900 raft_consensus.cc:2258] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:22:55.853237 25898 tablet_replica.cc:330] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae: stopping tablet replica
I20260811 22:22:55.853703 25898 raft_consensus.cc:2229] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:22:55.854856 25898 raft_consensus.cc:2258] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:22:55.857707 25900 ts_tablet_manager.cc:1905] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:22:55.858228 25897 tablet_replica.cc:330] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285: stopping tablet replica
I20260811 22:22:55.858655 25897 raft_consensus.cc:2229] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:22:55.859165 25897 raft_consensus.cc:2258] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:22:55.861054 25898 ts_tablet_manager.cc:1905] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:22:55.861219 25897 ts_tablet_manager.cc:1905] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:22:55.863734 25860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:55.841509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50112 (local address 127.29.229.1:33947)
0811 22:22:55.841744 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:55.841753 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:55.841777 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:22:55.841850 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:55.841858 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:55.841967 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:55.842087 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:55.844663 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.846221 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.848977 (+ 2756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:55.849959 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:55.850107 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:55.854277 (+ 4170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:55.854293 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:55.854310 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:55.854356 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:55.859091 (+ 4735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:55.860116 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:55.860126 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:55.860130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:55.860305 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:55.860829 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:55.860834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:55.860837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:55.861424 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:55.861766 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:55.863069 (+ 1303us) server_negotiation.cc:299] Negotiation successful
0811 22:22:55.863314 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:22:55.864804 25431 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:55.870376 25897 ts_tablet_manager.cc:1918] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:22:55.870584 25897 log.cc:1196] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/35d0e80468d74c55be594d752d307060
I20260811 22:22:55.871186 25897 ts_tablet_manager.cc:1939] T 35d0e80468d74c55be594d752d307060 P 2376c62f68ec40f7ac5993e89fd86285: Deleting consensus metadata
I20260811 22:22:55.873337 25118 catalog_manager.cc:4841] TS 2376c62f68ec40f7ac5993e89fd86285 (127.29.229.3:39467): tablet 35d0e80468d74c55be594d752d307060 (table test-table [id=e7ed4aed59684ca1956b76693db382ca]) successfully deleted
I20260811 22:22:55.873728 25898 ts_tablet_manager.cc:1918] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:22:55.873925 25898 log.cc:1196] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/35d0e80468d74c55be594d752d307060
I20260811 22:22:55.874420 25898 ts_tablet_manager.cc:1939] T 35d0e80468d74c55be594d752d307060 P 6b9d9dfba06945f88970c30debaee5ae: Deleting consensus metadata
I20260811 22:22:55.875942 25121 catalog_manager.cc:4841] TS 6b9d9dfba06945f88970c30debaee5ae (127.29.229.2:39553): tablet 35d0e80468d74c55be594d752d307060 (table test-table [id=e7ed4aed59684ca1956b76693db382ca]) successfully deleted
I20260811 22:22:55.985482 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 25195
W20260811 22:22:56.013697 25124 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.229.1:33947 (error 108)
I20260811 22:22:56.014001 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 25446
I20260811 22:22:56.016531 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:56.016027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33947 (local address 127.0.0.1:50116)
0811 22:22:56.016209 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:56.016331 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 22:22:56.016805 25124 catalog_manager.cc:4584] TS 454fa69cf4d844b79d7688db9d8e722b (127.29.229.1:33947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 35d0e80468d74c55be594d752d307060: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33947: connect: Connection refused (error 111)
I20260811 22:22:56.030267 25864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:56.014987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39844 (local address 127.29.229.2:39553)
0811 22:22:56.015705 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:56.015713 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:56.015728 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:22:56.015796 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:56.015800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:56.015889 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:56.015999 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:56.017798 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:56.018663 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:56.020023 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:56.020571 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:56.020714 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:56.023368 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:56.023399 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:56.023427 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:56.023487 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:56.027049 (+ 3562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:56.027958 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:56.027969 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:56.027976 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:56.028137 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:56.028588 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:56.028597 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:56.028600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:56.029001 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:56.029377 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:56.029656 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 22:22:56.029904 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549}
I20260811 22:22:56.031035 25634 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:56.070876 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:56.070261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33947 (local address 127.0.0.1:50120)
0811 22:22:56.070537 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:56.070696 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260811 22:22:56.072046 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:56.071609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33947 (local address 127.0.0.1:50128)
0811 22:22:56.071824 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:56.071923 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 22:22:56.132725 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:56.132139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33947 (local address 127.0.0.1:50130)
0811 22:22:56.132401 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:56.132564 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260811 22:22:56.133988 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:56.133513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33947 (local address 127.0.0.1:50144)
0811 22:22:56.133779 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:56.133872 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260811 22:22:56.154232 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 25446
I20260811 22:22:56.190230 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 25648
I20260811 22:22:56.207793 25865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:56.191131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53796 (local address 127.29.229.3:39467)
0811 22:22:56.191362 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 22:22:56.191370 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:22:56.191481 (+ 111us) server_negotiation.cc:407] Connection header received
0811 22:22:56.191709 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:22:56.191714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:22:56.191784 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:22:56.191904 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:22:56.193428 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:56.194167 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:56.195701 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:22:56.196104 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:22:56.196250 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:22:56.199552 (+ 3302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:22:56.199569 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:22:56.199586 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:22:56.199629 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:22:56.202784 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:56.204297 (+ 1513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:56.204315 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:56.204318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:56.204589 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:22:56.205082 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:22:56.205088 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:22:56.205092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:22:56.205975 (+ 883us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:22:56.206475 (+ 500us) server_negotiation.cc:1035] Waiting for connection context
0811 22:22:56.207111 (+ 636us) server_negotiation.cc:299] Negotiation successful
0811 22:22:56.207425 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 22:22:56.208626 25839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:56.214740 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:56.214163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33947 (local address 127.0.0.1:50148)
0811 22:22:56.214403 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:56.214517 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 22:22:56.216325 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:56.215793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33947 (local address 127.0.0.1:50164)
0811 22:22:56.216084 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:56.216187 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
I20260811 22:22:56.306350 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 25648
I20260811 22:22:56.330842 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 25085
I20260811 22:22:56.332758 25186 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:22:56.461185 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:22:56.460508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:33947 (local address 127.0.0.1:50172)
0811 22:22:56.460773 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 22:22:56.460927 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:33947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 22:22:56.461355 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 25085
I20260811 22:23:16.496639 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:45475
--webserver_interface=127.29.229.62
--webserver_port=41253
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:45475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:23:16.874922 25912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:16.875233 25912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:16.875402 25912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:16.882992 25912 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:23:16.883095 25912 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:23:16.883181 25912 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:23:16.883278 25912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:23:16.883354 25912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:16.883422 25912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:16.883484 25912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:23:16.883572 25912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:23:16.894032 25912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:45475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:45475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=41253
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:16.895433 25912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:16.896898 25912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:16.910852 25918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:16.910840 25919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:16.910847 25924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:16.911463 25912 server_base.cc:1034] running on GCE node
I20260811 22:23:16.912551 25912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:16.914105 25912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:16.915382 25912 hybrid_clock.cc:648] HybridClock initialized: now 1786486996915328 us; error 32 us; skew 500 ppm
I20260811 22:23:16.915753 25912 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:23:16 dist-test-slave-jq4t krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486996, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:16.923671 25912 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:23:16.926194 25912 webserver.cc:460] Webserver started at http://127.29.229.62:41253/ using document root <none> and password file <none>
I20260811 22:23:16.927290 25912 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:16.927448 25912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:16.936566 25912 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 22:23:16.940995 25935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:16.943354 25912 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260811 22:23:16.943588 25912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6"
format_stamp: "Formatted at 2026-08-11 22:22:30 on dist-test-slave-jq4t"
server_key: "b7aa420b88eac14b438c66c7ce86e61b"
server_key_iv: "90b12d4a88f80c6e71b51e48f4dab6db"
server_key_version: "encryptionkey@0"
I20260811 22:23:16.944938 25912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:16.987967 25912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:16.993152 25912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:16.993436 25912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:17.015394 25912 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:45475
I20260811 22:23:17.015443 26010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:45475 every 8 connection(s)
I20260811 22:23:17.017069 25912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:23:17.020838 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 25912
I20260811 22:23:17.021924 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:33947
--local_ip_for_outbound_sockets=127.29.229.1
--tserver_master_addrs=127.29.229.62:45475
--webserver_port=38211
--webserver_interface=127.29.229.1
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:23:17.026326 26011 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:23:17.031735 26011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6: Bootstrap starting.
I20260811 22:23:17.055884 26011 log.cc:824] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:17.075117 26011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:23:17.076016 26011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6: Bootstrap complete.
I20260811 22:23:17.088153 26011 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45475 } }
I20260811 22:23:17.089367 26011 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a3ecbba44a646a1bba5bb4f0fa56ba6, State: Initialized, Role: FOLLOWER
I20260811 22:23:17.090075 26011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45475 } }
I20260811 22:23:17.090332 26011 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:23:17.090463 26011 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:23:17.090670 26011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:23:17.094910 26011 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45475 } }
I20260811 22:23:17.095479 26011 leader_election.cc:302] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a3ecbba44a646a1bba5bb4f0fa56ba6; no voters:
I20260811 22:23:17.096217 26011 leader_election.cc:288] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 22:23:17.096374 26016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 22:23:17.097738 26016 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [term 3 LEADER]: Becoming Leader. State: Replica: 6a3ecbba44a646a1bba5bb4f0fa56ba6, State: Running, Role: LEADER
I20260811 22:23:17.098457 26016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45475 } }
I20260811 22:23:17.098676 26011 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:23:17.101262 26019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a3ecbba44a646a1bba5bb4f0fa56ba6. Latest consensus state: current_term: 3 leader_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45475 } } }
I20260811 22:23:17.101346 26017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3ecbba44a646a1bba5bb4f0fa56ba6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45475 } } }
I20260811 22:23:17.101698 26019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:17.101761 26017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:17.102679 26023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:23:17.125247 26023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:23:17.126976 26023 catalog_manager.cc:1259] Loaded cluster ID: a5362d30f0074d67a133a754b4584d98
I20260811 22:23:17.127089 26023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:23:17.131418 26023 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:23:17.133279 26023 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6: Loaded TSK: 0
I20260811 22:23:17.133525 26023 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6a3ecbba44a646a1bba5bb4f0fa56ba6: Loaded TSK: 1
F20260811 22:23:17.135227 26023 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6e85b9e9ad google::LogMessage::Fail() at ??:0
@ 0x7f6e85ba188a google::LogMessage::SendToLog() at ??:0
@ 0x7f6e85b9e4de google::LogMessage::Flush() at ??:0
@ 0x7f6e85b9fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6e899027d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f6e898f173e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f6e9b5cd1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f6e9b5cc783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f6e9b6582a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f6e9b6580c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f6e9b6a25e9 std::function<>::operator()() at ??:0
@ 0x7f6e9b5d0cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f6e9b5cec36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6e9b656610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f6e9b656439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6e887ea345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6e887ed91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f6e887ed799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6e887c8eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6e8baa26db start_thread at ??:0
@ 0x7f6e84a3861f clone at ??:0
W20260811 22:23:17.499976 26014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:17.500298 26014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:17.500444 26014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:17.508637 26014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:17.508770 26014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:17.508957 26014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:23:17.520913 26014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:33947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=38211
--tserver_master_addrs=127.29.229.62:45475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:17.522308 26014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:17.523702 26014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:17.540798 26075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:17.541014 26081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:17.541043 26079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:17.541998 26014 server_base.cc:1034] running on GCE node
I20260811 22:23:17.542985 26014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:17.544337 26014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:17.545881 26014 hybrid_clock.cc:648] HybridClock initialized: now 1786486997545522 us; error 71 us; skew 500 ppm
I20260811 22:23:17.546289 26014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:23:17 dist-test-slave-jq4t krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486997, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:17.550086 26014 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:23:17.552096 26014 webserver.cc:460] Webserver started at http://127.29.229.1:38211/ using document root <none> and password file <none>
I20260811 22:23:17.552989 26014 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:17.553158 26014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:17.559008 26014 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 22:23:17.562377 26089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:17.563870 26014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:23:17.564071 26014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "454fa69cf4d844b79d7688db9d8e722b"
format_stamp: "Formatted at 2026-08-11 22:22:30 on dist-test-slave-jq4t"
server_key: "e0f3998d51d997339b594aaa503b1e17"
server_key_iv: "6f9bef3f96de25cd0a983527762693dd"
server_key_version: "encryptionkey@0"
I20260811 22:23:17.565171 26014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:17.588161 26014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:17.592898 26014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:17.593206 26014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:17.594286 26014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:17.596233 26099 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 22:23:17.602144 26014 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:23:17.602310 26014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260811 22:23:17.602473 26014 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:23:17.605852 26014 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:23:17.605963 26014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.002s sys 0.000s
I20260811 22:23:17.606285 26099 tablet_bootstrap.cc:492] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b: Bootstrap starting.
I20260811 22:23:17.615157 26099 log.cc:824] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:17.635504 26099 tablet_bootstrap.cc:492] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:23:17.636343 26099 tablet_bootstrap.cc:492] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b: Bootstrap complete.
I20260811 22:23:17.637048 26099 ts_tablet_manager.cc:1397] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b: Time spent bootstrapping tablet: real 0.031s user 0.029s sys 0.000s
I20260811 22:23:17.646458 26099 raft_consensus.cc:348] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } }
I20260811 22:23:17.648233 26099 raft_consensus.cc:729] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 454fa69cf4d844b79d7688db9d8e722b, State: Initialized, Role: FOLLOWER
I20260811 22:23:17.649379 26099 consensus_queue.cc:260] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } }
I20260811 22:23:17.651641 26099 ts_tablet_manager.cc:1428] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260811 22:23:17.653781 26014 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:33947
I20260811 22:23:17.653822 26284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:33947 every 8 connection(s)
I20260811 22:23:17.655154 26014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:23:17.660295 26290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:17.658292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:37409)
0811 22:23:17.659111 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:17.659401 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":334,"threads_started":1}
W20260811 22:23:17.661023 26285 heartbeater.cc:644] Failed to heartbeat to 127.29.229.62:45475 (0 consecutive failures): Network error: Failed to ping master at 127.29.229.62:45475: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
I20260811 22:23:17.661912 26290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:17.661603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:35773)
0811 22:23:17.661743 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:17.661810 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 22:23:17.662748 26290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:17.662439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:45039)
0811 22:23:17.662568 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:17.662635 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:23:17.663028 26285 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:23:17.665594 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 26014
I20260811 22:23:17.666255 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:39553
--local_ip_for_outbound_sockets=127.29.229.2
--tserver_master_addrs=127.29.229.62:45475
--webserver_port=44573
--webserver_interface=127.29.229.2
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:23:18.037467 26293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:18.037771 26293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:18.037881 26293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:18.046197 26293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:18.046401 26293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:18.046561 26293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:23:18.058427 26293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:39553
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=44573
--tserver_master_addrs=127.29.229.62:45475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:18.060031 26293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:18.061486 26293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:18.075840 26305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:18.075892 26303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:18.075870 26307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:18.077562 26293 server_base.cc:1034] running on GCE node
I20260811 22:23:18.078289 26293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:18.079385 26293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:18.080755 26293 hybrid_clock.cc:648] HybridClock initialized: now 1786486998080641 us; error 56 us; skew 500 ppm
I20260811 22:23:18.081408 26293 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:23:18 dist-test-slave-jq4t krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486998, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:18.085567 26293 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:23:18.087229 26293 webserver.cc:460] Webserver started at http://127.29.229.2:44573/ using document root <none> and password file <none>
I20260811 22:23:18.087960 26293 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:18.088078 26293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:18.094089 26293 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 22:23:18.097347 26322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:18.099191 26293 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260811 22:23:18.099377 26293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "6b9d9dfba06945f88970c30debaee5ae"
format_stamp: "Formatted at 2026-08-11 22:22:31 on dist-test-slave-jq4t"
server_key: "0b4fa2daba7d77bed1e8c60374d378d9"
server_key_iv: "59379c81bfd6c61a184b79d6ee436e13"
server_key_version: "encryptionkey@0"
I20260811 22:23:18.100414 26293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:18.117852 26293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:18.122991 26293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:18.123251 26293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:18.124362 26293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:18.125888 26293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:18.125993 26293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:18.126123 26293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:18.126204 26293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:18.167371 26293 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:39553
I20260811 22:23:18.167503 26507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:39553 every 8 connection(s)
I20260811 22:23:18.169095 26293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:23:18.174360 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:18.172403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:57031)
0811 22:23:18.173251 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:18.173496 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":318,"threads_started":1}
W20260811 22:23:18.175293 26508 heartbeater.cc:644] Failed to heartbeat to 127.29.229.62:45475 (0 consecutive failures): Network error: Failed to ping master at 127.29.229.62:45475: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
I20260811 22:23:18.176090 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 26293
I20260811 22:23:18.176447 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:18.176010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:45179)
0811 22:23:18.176235 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:18.176325 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 22:23:18.176864 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:39467
--local_ip_for_outbound_sockets=127.29.229.3
--tserver_master_addrs=127.29.229.62:45475
--webserver_port=40113
--webserver_interface=127.29.229.3
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:46537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:23:18.177522 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:18.177168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:45357)
0811 22:23:18.177316 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:18.177406 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 22:23:18.177757 26508 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:23:18.548462 26516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:18.548836 26516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:18.549008 26516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:18.556320 26516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:18.556433 26516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:18.556676 26516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:23:18.566635 26516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:39467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=40113
--tserver_master_addrs=127.29.229.62:45475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:18.567987 26516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:18.569454 26516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:18.583199 26524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:18.583148 26526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:18.583232 26529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:18.584340 26516 server_base.cc:1034] running on GCE node
I20260811 22:23:18.585006 26516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:18.586014 26516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:18.587222 26516 hybrid_clock.cc:648] HybridClock initialized: now 1786486998587169 us; error 48 us; skew 500 ppm
I20260811 22:23:18.587563 26516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:23:18 dist-test-slave-jq4t krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486998, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:18.591009 26516 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:23:18.593255 26516 webserver.cc:460] Webserver started at http://127.29.229.3:40113/ using document root <none> and password file <none>
I20260811 22:23:18.594048 26516 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:18.594165 26516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:18.599890 26516 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 22:23:18.602931 26539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:18.604417 26516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:23:18.604699 26516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "2376c62f68ec40f7ac5993e89fd86285"
format_stamp: "Formatted at 2026-08-11 22:22:32 on dist-test-slave-jq4t"
server_key: "3cdd2c34aac8a9b29fab5410d26ec1f3"
server_key_iv: "f9153a48dce6b669fc80a244da0788c3"
server_key_version: "encryptionkey@0"
I20260811 22:23:18.605815 26516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:18.632970 26516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:18.637495 26516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:18.637800 26516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:18.638914 26516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:18.640380 26516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:18.640530 26516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:18.640673 26516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:18.640784 26516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:18.665591 26642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:18.663855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:60575)
0811 22:23:18.665141 (+ 1286us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:18.665272 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1146,"thread_start_us":947,"threads_started":1}
I20260811 22:23:18.682348 26516 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:39467
I20260811 22:23:18.682440 26738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:39467 every 8 connection(s)
I20260811 22:23:18.683872 26516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:23:18.687593 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 26516
I20260811 22:23:18.689944 26741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:18.686908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:56007)
0811 22:23:18.687904 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:18.688208 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":373,"threads_started":1}
W20260811 22:23:18.690769 26739 heartbeater.cc:644] Failed to heartbeat to 127.29.229.62:45475 (0 consecutive failures): Network error: Failed to ping master at 127.29.229.62:45475: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
I20260811 22:23:18.692145 26741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:18.691777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:58927)
0811 22:23:18.691944 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:18.692023 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 22:23:18.693262 26741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:18.692874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:37233)
0811 22:23:18.693050 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:18.693145 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260811 22:23:18.693567 26739 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:23:19.179260 26744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:19.178368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:52041)
0811 22:23:19.178900 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:19.179015 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":269,"threads_started":1}
I20260811 22:23:19.281571 26745 raft_consensus.cc:482] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:23:19.281889 26745 raft_consensus.cc:504] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } }
I20260811 22:23:19.283874 26745 leader_election.cc:288] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2376c62f68ec40f7ac5993e89fd86285 (127.29.229.3:39467), 6b9d9dfba06945f88970c30debaee5ae (127.29.229.2:39553)
Aug 11 22:23:19 dist-test-slave-jq4t krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486997, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.2@KRBTEST.COM
Aug 11 22:23:19 dist-test-slave-jq4t krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486997, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.3@KRBTEST.COM
I20260811 22:23:19.305723 26746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:19.283932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:39467 (local address 127.29.229.1:44337)
0811 22:23:19.284652 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:19.284699 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:19.285299 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:19.288854 (+ 3555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:19.288866 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:19.289668 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:19.290313 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:19.290333 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:19.291931 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:19.291935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:19.293390 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:19.293404 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:19.294650 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:19.294659 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:19.294880 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:19.295686 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:19.295728 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:19.298589 (+ 2861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:19.301995 (+ 3406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:19.302000 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:19.302004 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:19.302469 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:19.302920 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:19.302925 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:19.302929 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:19.303173 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:19.303966 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:19.303974 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:19.304324 (+ 350us) client_negotiation.cc:769] Sending connection context
0811 22:23:19.304641 (+ 317us) client_negotiation.cc:240] Negotiation successful
0811 22:23:19.305220 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"mutex_wait_us":140,"spinlock_wait_cycles":31360,"thread_start_us":325,"threads_started":1}
I20260811 22:23:19.305750 26748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:19.284358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:44337 (local address 127.29.229.3:39467)
0811 22:23:19.286197 (+ 1839us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:19.286219 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:19.286268 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:23:19.288000 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:19.288009 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:19.288356 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:19.288739 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:19.290531 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:19.291727 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:19.293549 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:19.294438 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:19.294650 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:19.298872 (+ 4222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:19.298879 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:19.298890 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:19.298928 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:19.301843 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:19.302619 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:19.302625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:19.302630 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:19.302788 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:19.303297 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:19.303304 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:19.303312 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:19.303795 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:19.304063 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:19.304877 (+ 814us) server_negotiation.cc:299] Negotiation successful
0811 22:23:19.305341 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1612,"thread_start_us":1478,"threads_started":1}
I20260811 22:23:19.306193 26749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:19.284358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:39553 (local address 127.29.229.1:33617)
0811 22:23:19.286582 (+ 2224us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:19.286606 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:19.286747 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:19.289346 (+ 2599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:19.289351 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:19.290051 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:19.290474 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:19.290484 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:19.292141 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:19.292145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:19.293749 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:19.293766 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:19.295113 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:19.295120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:19.295307 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:19.295823 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:19.295840 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:19.298293 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:19.301910 (+ 3617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:19.301919 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:19.301934 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:19.302467 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:19.302898 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:19.302906 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:19.302910 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:19.303128 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:19.303997 (+ 869us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:19.304002 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:19.304324 (+ 322us) client_negotiation.cc:769] Sending connection context
0811 22:23:19.304643 (+ 319us) client_negotiation.cc:240] Negotiation successful
0811 22:23:19.305761 (+ 1118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2076,"mutex_wait_us":241,"thread_start_us":1851,"threads_started":1}
I20260811 22:23:19.305754 26750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:19.284926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:33617 (local address 127.29.229.2:39553)
0811 22:23:19.286975 (+ 2049us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:19.286985 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:19.287020 (+ 35us) server_negotiation.cc:407] Connection header received
0811 22:23:19.288732 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:19.288741 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:19.289056 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:19.289363 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:19.290670 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:19.291946 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:19.293921 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:19.294892 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:19.295111 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:19.298605 (+ 3494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:19.298613 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:19.298627 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:19.298663 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:19.301731 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:19.302616 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:19.302623 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:19.302628 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:19.302778 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:19.303257 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:19.303261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:19.303263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:19.303795 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:19.304007 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:19.304876 (+ 869us) server_negotiation.cc:299] Negotiation successful
0811 22:23:19.305365 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1875,"thread_start_us":1697,"threads_started":1}
W20260811 22:23:19.309767 26091 leader_election.cc:341] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6b9d9dfba06945f88970c30debaee5ae (127.29.229.2:39553): Not found: Tablet not found: 35d0e80468d74c55be594d752d307060
W20260811 22:23:19.310403 26090 leader_election.cc:341] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2376c62f68ec40f7ac5993e89fd86285 (127.29.229.3:39467): Not found: Tablet not found: 35d0e80468d74c55be594d752d307060
I20260811 22:23:19.310928 26090 leader_election.cc:302] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 454fa69cf4d844b79d7688db9d8e722b; no voters: 2376c62f68ec40f7ac5993e89fd86285, 6b9d9dfba06945f88970c30debaee5ae
I20260811 22:23:19.311650 26745 raft_consensus.cc:2726] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:23:19.667331 26746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:19.666882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:55719)
0811 22:23:19.667085 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:19.667182 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 22:23:19.695559 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:19.694431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:34737)
0811 22:23:19.695098 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:19.695292 (+ 194us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":355,"threads_started":1}
I20260811 22:23:20.181154 26756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:20.180334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:47733)
0811 22:23:20.180891 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:20.180983 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":349,"threads_started":1}
I20260811 22:23:20.669656 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:20.668382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:45819)
0811 22:23:20.669201 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:20.669384 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":471,"threads_started":1}
I20260811 22:23:20.697427 26759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:20.696634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:56997)
0811 22:23:20.697180 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:20.697261 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":324,"threads_started":1}
I20260811 22:23:20.977159 26763 raft_consensus.cc:482] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:23:20.977453 26763 raft_consensus.cc:504] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } }
I20260811 22:23:20.978245 26763 leader_election.cc:288] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2376c62f68ec40f7ac5993e89fd86285 (127.29.229.3:39467), 6b9d9dfba06945f88970c30debaee5ae (127.29.229.2:39553)
W20260811 22:23:20.979466 26091 leader_election.cc:341] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6b9d9dfba06945f88970c30debaee5ae (127.29.229.2:39553): Not found: Tablet not found: 35d0e80468d74c55be594d752d307060
W20260811 22:23:20.979705 26090 leader_election.cc:341] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2376c62f68ec40f7ac5993e89fd86285 (127.29.229.3:39467): Not found: Tablet not found: 35d0e80468d74c55be594d752d307060
I20260811 22:23:20.979842 26090 leader_election.cc:302] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 454fa69cf4d844b79d7688db9d8e722b; no voters: 2376c62f68ec40f7ac5993e89fd86285, 6b9d9dfba06945f88970c30debaee5ae
I20260811 22:23:20.980167 26763 raft_consensus.cc:2726] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:23:21.183195 26764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:21.182333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:33253)
0811 22:23:21.182880 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:21.182979 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":297,"threads_started":1}
I20260811 22:23:21.671698 26767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:21.670767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:42641)
0811 22:23:21.671370 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:21.671480 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":336,"threads_started":1}
I20260811 22:23:21.699766 26768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:21.698783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:59305)
0811 22:23:21.699466 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:21.699583 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":388,"threads_started":1}
I20260811 22:23:22.185464 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:22.184380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:43363)
0811 22:23:22.185087 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:22.185203 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":408,"threads_started":1}
I20260811 22:23:22.484171 26771 raft_consensus.cc:482] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:23:22.484421 26771 raft_consensus.cc:504] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } }
I20260811 22:23:22.485239 26771 leader_election.cc:288] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2376c62f68ec40f7ac5993e89fd86285 (127.29.229.3:39467), 6b9d9dfba06945f88970c30debaee5ae (127.29.229.2:39553)
W20260811 22:23:22.486284 26090 leader_election.cc:341] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2376c62f68ec40f7ac5993e89fd86285 (127.29.229.3:39467): Not found: Tablet not found: 35d0e80468d74c55be594d752d307060
W20260811 22:23:22.486483 26091 leader_election.cc:341] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6b9d9dfba06945f88970c30debaee5ae (127.29.229.2:39553): Not found: Tablet not found: 35d0e80468d74c55be594d752d307060
I20260811 22:23:22.486652 26091 leader_election.cc:302] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 454fa69cf4d844b79d7688db9d8e722b; no voters: 2376c62f68ec40f7ac5993e89fd86285, 6b9d9dfba06945f88970c30debaee5ae
I20260811 22:23:22.487008 26771 raft_consensus.cc:2726] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:23:22.673825 26772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:22.672814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:60093)
0811 22:23:22.673436 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:22.673566 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":319,"threads_started":1}
I20260811 22:23:22.701959 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:22.701127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:36861)
0811 22:23:22.701686 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:22.701768 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":323,"threads_started":1}
I20260811 22:23:23.187659 26774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:23.186902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:45831)
0811 22:23:23.187387 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:23.187476 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":241,"threads_started":1}
I20260811 22:23:23.675536 26778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:23.674820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:59987)
0811 22:23:23.675268 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:23.675371 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":235,"threads_started":1}
I20260811 22:23:23.703780 26782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:23.702979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:42023)
0811 22:23:23.703519 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:23.703617 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":323,"threads_started":1}
I20260811 22:23:24.189815 26783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:24.188920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:33431)
0811 22:23:24.189511 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:24.189605 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":338,"threads_started":1}
I20260811 22:23:24.677838 26784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:24.676744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:43703)
0811 22:23:24.677400 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:24.677546 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":358,"threads_started":1}
I20260811 22:23:24.705760 26785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:24.704944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:49871)
0811 22:23:24.705489 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:24.705578 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":315,"threads_started":1}
I20260811 22:23:25.192728 26786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:25.191479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:33757)
0811 22:23:25.192337 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:25.192446 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":395,"threads_started":1}
I20260811 22:23:25.679754 26787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:25.678951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:33069)
0811 22:23:25.679459 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:25.679563 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":274,"threads_started":1}
I20260811 22:23:25.707724 26788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:25.706828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:55411)
0811 22:23:25.707445 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:25.707526 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":316,"threads_started":1}
I20260811 22:23:26.194931 26792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:26.194069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:57285)
0811 22:23:26.194612 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:26.194750 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":322,"threads_started":1}
I20260811 22:23:26.304191 26793 raft_consensus.cc:482] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:23:26.304423 26793 raft_consensus.cc:504] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } }
I20260811 22:23:26.305233 26793 leader_election.cc:288] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2376c62f68ec40f7ac5993e89fd86285 (127.29.229.3:39467), 6b9d9dfba06945f88970c30debaee5ae (127.29.229.2:39553)
W20260811 22:23:26.306391 26091 leader_election.cc:341] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6b9d9dfba06945f88970c30debaee5ae (127.29.229.2:39553): Not found: Tablet not found: 35d0e80468d74c55be594d752d307060
W20260811 22:23:26.306629 26090 leader_election.cc:341] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2376c62f68ec40f7ac5993e89fd86285 (127.29.229.3:39467): Not found: Tablet not found: 35d0e80468d74c55be594d752d307060
I20260811 22:23:26.306802 26090 leader_election.cc:302] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 454fa69cf4d844b79d7688db9d8e722b; no voters: 2376c62f68ec40f7ac5993e89fd86285, 6b9d9dfba06945f88970c30debaee5ae
I20260811 22:23:26.307070 26793 raft_consensus.cc:2726] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:23:26.681735 26794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:26.680879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:45711)
0811 22:23:26.681429 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:26.681536 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":288,"threads_started":1}
I20260811 22:23:26.709815 26796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:26.708848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:59733)
0811 22:23:26.709540 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:26.709625 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":417,"threads_started":1}
I20260811 22:23:27.197530 26797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:27.196360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:32849)
0811 22:23:27.197196 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:27.197354 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":375,"threads_started":1}
I20260811 22:23:27.683692 26798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:27.682954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:60341)
0811 22:23:27.683436 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:27.683538 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":277,"threads_started":1}
I20260811 22:23:27.712445 26802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:27.711437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:45881)
0811 22:23:27.712145 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:27.712258 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":382,"threads_started":1}
I20260811 22:23:28.122350 26803 raft_consensus.cc:482] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:23:28.122637 26803 raft_consensus.cc:504] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } }
I20260811 22:23:28.123560 26803 leader_election.cc:288] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2376c62f68ec40f7ac5993e89fd86285 (127.29.229.3:39467), 6b9d9dfba06945f88970c30debaee5ae (127.29.229.2:39553)
W20260811 22:23:28.124621 26090 leader_election.cc:341] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2376c62f68ec40f7ac5993e89fd86285 (127.29.229.3:39467): Not found: Tablet not found: 35d0e80468d74c55be594d752d307060
W20260811 22:23:28.124876 26091 leader_election.cc:341] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6b9d9dfba06945f88970c30debaee5ae (127.29.229.2:39553): Not found: Tablet not found: 35d0e80468d74c55be594d752d307060
I20260811 22:23:28.125037 26091 leader_election.cc:302] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 454fa69cf4d844b79d7688db9d8e722b; no voters: 2376c62f68ec40f7ac5993e89fd86285, 6b9d9dfba06945f88970c30debaee5ae
I20260811 22:23:28.125312 26803 raft_consensus.cc:2726] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:23:28.199743 26806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:28.198647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:33297)
0811 22:23:28.199339 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:28.199478 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":346,"threads_started":1}
I20260811 22:23:28.685501 26807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:28.684663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:60753)
0811 22:23:28.685229 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:28.685315 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":331,"threads_started":1}
I20260811 22:23:28.714306 26809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:28.713470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:34227)
0811 22:23:28.714013 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:28.714113 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":254,"threads_started":1}
I20260811 22:23:29.202039 26810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:29.201163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:36293)
0811 22:23:29.201748 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:29.201853 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":294,"threads_started":1}
I20260811 22:23:29.687301 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:29.686465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:42115)
0811 22:23:29.687038 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:29.687147 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":349,"threads_started":1}
I20260811 22:23:29.716074 26814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:29.715259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:35573)
0811 22:23:29.715825 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:29.715913 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":299,"threads_started":1}
I20260811 22:23:30.031184 26815 raft_consensus.cc:482] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:23:30.031409 26815 raft_consensus.cc:504] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454fa69cf4d844b79d7688db9d8e722b" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 33947 } } peers { permanent_uuid: "2376c62f68ec40f7ac5993e89fd86285" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 39467 } } peers { permanent_uuid: "6b9d9dfba06945f88970c30debaee5ae" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 39553 } }
I20260811 22:23:30.032402 26815 leader_election.cc:288] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2376c62f68ec40f7ac5993e89fd86285 (127.29.229.3:39467), 6b9d9dfba06945f88970c30debaee5ae (127.29.229.2:39553)
W20260811 22:23:30.033468 26091 leader_election.cc:341] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6b9d9dfba06945f88970c30debaee5ae (127.29.229.2:39553): Not found: Tablet not found: 35d0e80468d74c55be594d752d307060
W20260811 22:23:30.033704 26090 leader_election.cc:341] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2376c62f68ec40f7ac5993e89fd86285 (127.29.229.3:39467): Not found: Tablet not found: 35d0e80468d74c55be594d752d307060
I20260811 22:23:30.033865 26090 leader_election.cc:302] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 454fa69cf4d844b79d7688db9d8e722b; no voters: 2376c62f68ec40f7ac5993e89fd86285, 6b9d9dfba06945f88970c30debaee5ae
I20260811 22:23:30.034132 26815 raft_consensus.cc:2726] T 35d0e80468d74c55be594d752d307060 P 454fa69cf4d844b79d7688db9d8e722b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:23:30.204002 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:30.203189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:42067)
0811 22:23:30.203722 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:30.203816 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":305,"threads_started":1}
I20260811 22:23:30.689153 26817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:30.688391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:60333)
0811 22:23:30.688876 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:30.688977 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":269,"threads_started":1}
I20260811 22:23:30.718422 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:30.717562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:56173)
0811 22:23:30.718154 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:30.718239 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":342,"threads_started":1}
I20260811 22:23:31.206308 26820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:31.205409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:33029)
0811 22:23:31.206016 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:31.206143 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":338,"threads_started":1}
I20260811 22:23:31.691092 26821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:31.690033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:46931)
0811 22:23:31.690693 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:31.690840 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":332,"threads_started":1}
I20260811 22:23:31.720574 26822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:31.719569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:37389)
0811 22:23:31.720185 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:31.720291 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":375,"threads_started":1}
I20260811 22:23:32.208175 26823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:32.207308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:42929)
0811 22:23:32.207869 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:32.207960 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":263,"threads_started":1}
I20260811 22:23:32.693372 26824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:32.692559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:57843)
0811 22:23:32.693089 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:32.693181 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":269,"threads_started":1}
I20260811 22:23:32.722352 26825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:32.721501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:35507)
0811 22:23:32.722042 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:32.722126 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":267,"threads_started":1}
I20260811 22:23:33.211149 26826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:33.210106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.2:55719)
0811 22:23:33.210864 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:33.210982 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":470,"threads_started":1}
I20260811 22:23:33.695248 26828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:33.694520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.1:60621)
0811 22:23:33.694999 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:33.695087 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":230,"threads_started":1}
I20260811 22:23:33.697134 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 26014
I20260811 22:23:33.716738 26829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:33.698146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51128 (local address 127.29.229.1:33947)
0811 22:23:33.699301 (+ 1155us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:33.699308 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:33.699335 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:23:33.699399 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:33.699405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:33.699611 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:33.699744 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:33.701398 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:33.702507 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:33.703745 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:33.704715 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:33.704869 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:33.710887 (+ 6018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:33.710900 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:33.710916 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:33.710959 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:33.713968 (+ 3009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:33.714653 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:33.714661 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:33.714664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:33.714833 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:33.715223 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:33.715227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:33.715230 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:33.715602 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:33.715862 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:33.716182 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 22:23:33.716388 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":982,"thread_start_us":831,"threads_started":1}
I20260811 22:23:33.718072 26269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:33.724371 26832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:33.723622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:45475 (local address 127.29.229.3:55533)
0811 22:23:33.724107 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:33.724224 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.62:45475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":268,"threads_started":1}
I20260811 22:23:33.812898 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 26014
I20260811 22:23:33.845091 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 26293
I20260811 22:23:33.860631 26833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:33.845933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43200 (local address 127.29.229.2:39553)
0811 22:23:33.846492 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:33.846496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:33.846515 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:23:33.846582 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:33.846585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:33.846645 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:33.846745 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:33.848370 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:33.849311 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:33.850724 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:33.851180 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:33.851291 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:33.854462 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:33.854470 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:33.854475 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:33.854535 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:33.857633 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:33.858361 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:33.858366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:33.858369 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:33.858499 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:33.858995 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:33.858999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:33.859002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:33.859310 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:33.859551 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:33.859921 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 22:23:33.860104 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":307,"threads_started":1}
I20260811 22:23:33.861485 26501 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:33.948707 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 26293
I20260811 22:23:33.979553 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 26516
I20260811 22:23:33.994935 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:33.980687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46670 (local address 127.29.229.3:39467)
0811 22:23:33.981463 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:33.981468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:33.981488 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:23:33.981547 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:33.981551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:33.981624 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:33.981729 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:33.983205 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:33.984019 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:33.985450 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:33.985880 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:33.985981 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:33.988658 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:33.988663 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:33.988666 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:33.988699 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:33.991554 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:33.992321 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:33.992325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:33.992328 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:33.992729 (+ 401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:33.993451 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:33.993455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:33.993457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:33.993766 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:33.994005 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:33.994418 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 22:23:33.994626 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":331,"threads_started":1}
I20260811 22:23:33.995729 26733 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:34.084000 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 26516
2026-08-11T22:23:34Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (66046 ms)
[----------] 30 tests from SecurityITest (328928 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:23:34 dist-test-slave-jq4t krb5kdc[26842](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:23:34 dist-test-slave-jq4t krb5kdc[26842](info): set up 2 sockets
Aug 11 22:23:34 dist-test-slave-jq4t krb5kdc[26842](info): commencing operation
krb5kdc: starting...
W20260811 22:23:35.511824 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.377s user 0.000s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:23:35 dist-test-slave-jq4t krb5kdc[26842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:23:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:23:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:23:35.889729 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:40327
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:37729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:40327
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:23:36.288677 26865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:36.289047 26865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:36.289254 26865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:36.297140 26865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:23:36.297273 26865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:36.297335 26865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:36.297458 26865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:23:36.297559 26865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:23:36.308291 26865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:37729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:40327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:40327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:36.309852 26865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:36.311308 26865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:36.324189 26875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:36.324134 26871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:36.324154 26872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:36.325022 26865 server_base.cc:1034] running on GCE node
I20260811 22:23:36.325996 26865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:36.327677 26865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:36.328965 26865 hybrid_clock.cc:648] HybridClock initialized: now 1786487016328911 us; error 43 us; skew 500 ppm
I20260811 22:23:36.329375 26865 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:23:36 dist-test-slave-jq4t krb5kdc[26842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487016, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:36.333001 26865 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:23:36.334887 26865 webserver.cc:460] Webserver started at http://127.29.229.62:35245/ using document root <none> and password file <none>
I20260811 22:23:36.335664 26865 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:36.335831 26865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:36.336167 26865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:36.340209 26865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "d761263c466249b1887fded9096b5379"
format_stamp: "Formatted at 2026-08-11 22:23:36 on dist-test-slave-jq4t"
server_key: "654b2a8b63f945a26111982b93a94e77"
server_key_iv: "5e02687758c87bfb22de688926ce7314"
server_key_version: "encryptionkey@0"
I20260811 22:23:36.341454 26865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "d761263c466249b1887fded9096b5379"
format_stamp: "Formatted at 2026-08-11 22:23:36 on dist-test-slave-jq4t"
server_key: "654b2a8b63f945a26111982b93a94e77"
server_key_iv: "5e02687758c87bfb22de688926ce7314"
server_key_version: "encryptionkey@0"
I20260811 22:23:36.348845 26865 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260811 22:23:36.353555 26883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:36.355428 26865 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:23:36.355657 26865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "d761263c466249b1887fded9096b5379"
format_stamp: "Formatted at 2026-08-11 22:23:36 on dist-test-slave-jq4t"
server_key: "654b2a8b63f945a26111982b93a94e77"
server_key_iv: "5e02687758c87bfb22de688926ce7314"
server_key_version: "encryptionkey@0"
I20260811 22:23:36.355898 26865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:36.374106 26865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:36.378855 26865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:36.379251 26865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:36.403054 26865 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:40327
I20260811 22:23:36.403057 26965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:40327 every 8 connection(s)
I20260811 22:23:36.404695 26865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:23:36.406266 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 26865
I20260811 22:23:36.406495 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:23:36.407217 30612 external_mini_cluster.cc:1428] Setting key 4f6100a149d36f884b3bb201b983645d
I20260811 22:23:36.409775 26966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:36.422331 26966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379: Bootstrap starting.
Aug 11 22:23:36 dist-test-slave-jq4t krb5kdc[26842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:23:36.426576 26966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:36.427757 26966 log.cc:824] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:36.432204 26966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379: No bootstrap required, opened a new log
I20260811 22:23:36.433812 26969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:36.409886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38846 (local address 127.29.229.62:40327)
0811 22:23:36.411105 (+ 1219us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:36.411120 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:36.411168 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:23:36.413007 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:36.413031 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:36.413472 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:36.413911 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:36.415522 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:36.416987 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:36.418404 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:36.419169 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:36.419380 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:36.423946 (+ 4566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:36.423954 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:36.423973 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:36.424029 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:36.427749 (+ 3720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:36.428637 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:36.428644 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:36.428650 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:36.428831 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:36.429368 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:36.429373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:36.429376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:36.430736 (+ 1360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:36.431130 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:36.431886 (+ 756us) server_negotiation.cc:299] Negotiation successful
0811 22:23:36.432384 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":969,"thread_start_us":609,"threads_started":1}
I20260811 22:23:36.440676 26966 raft_consensus.cc:348] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d761263c466249b1887fded9096b5379" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40327 } }
I20260811 22:23:36.441063 26966 raft_consensus.cc:374] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:36.441421 26966 raft_consensus.cc:729] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d761263c466249b1887fded9096b5379, State: Initialized, Role: FOLLOWER
I20260811 22:23:36.442107 26966 consensus_queue.cc:260] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d761263c466249b1887fded9096b5379" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40327 } }
I20260811 22:23:36.442378 26966 raft_consensus.cc:388] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:23:36.442534 26966 raft_consensus.cc:482] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:23:36.442727 26966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:36.445251 26966 raft_consensus.cc:504] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d761263c466249b1887fded9096b5379" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40327 } }
I20260811 22:23:36.445840 26966 leader_election.cc:302] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d761263c466249b1887fded9096b5379; no voters:
I20260811 22:23:36.446529 26966 leader_election.cc:288] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:23:36.446733 26977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:23:36.447739 26977 raft_consensus.cc:686] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379 [term 1 LEADER]: Becoming Leader. State: Replica: d761263c466249b1887fded9096b5379, State: Running, Role: LEADER
I20260811 22:23:36.448357 26977 consensus_queue.cc:237] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d761263c466249b1887fded9096b5379" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40327 } }
I20260811 22:23:36.449232 26966 sys_catalog.cc:564] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:23:36.453073 26979 sys_catalog.cc:455] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d761263c466249b1887fded9096b5379. Latest consensus state: current_term: 1 leader_uuid: "d761263c466249b1887fded9096b5379" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d761263c466249b1887fded9096b5379" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40327 } } }
I20260811 22:23:36.453611 26979 sys_catalog.cc:458] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379 [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:36.454092 26978 sys_catalog.cc:455] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d761263c466249b1887fded9096b5379" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d761263c466249b1887fded9096b5379" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40327 } } }
I20260811 22:23:36.460279 26978 sys_catalog.cc:458] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379 [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:36.463687 27003 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 22:23:36.465102 26993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:23:36.469381 26993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:23:36.478365 26993 catalog_manager.cc:1347] Generated new cluster ID: cf35a2150e564d1e807078b348728cdf
I20260811 22:23:36.478498 26993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:23:36.497916 26993 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:23:36.499488 26993 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:23:36.507850 26993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379: Generated new TSK 0
I20260811 22:23:36.509172 26993 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
I20260811 22:23:36.605031 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:40327
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:37729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:23:36.995404 27010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:36.995704 27010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:36.995868 27010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:37.003427 27010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:37.003543 27010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:37.003757 27010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:23:37.014406 27010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:37729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:40327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:37.015799 27010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:37.017227 27010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:37.030697 27017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:37.030761 27019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:37.030782 27016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:37.032240 27010 server_base.cc:1034] running on GCE node
I20260811 22:23:37.032995 27010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:37.034149 27010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:37.035392 27010 hybrid_clock.cc:648] HybridClock initialized: now 1786487017035337 us; error 40 us; skew 500 ppm
I20260811 22:23:37.035729 27010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:23:37 dist-test-slave-jq4t krb5kdc[26842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487017, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:37.039222 27010 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:23:37.040970 27010 webserver.cc:460] Webserver started at http://127.29.229.1:37955/ using document root <none> and password file <none>
I20260811 22:23:37.041679 27010 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:37.041788 27010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:37.042065 27010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:37.046586 27010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "7acf6ddb739f49a88138f226eea2c915"
format_stamp: "Formatted at 2026-08-11 22:23:37 on dist-test-slave-jq4t"
server_key: "52dbad3e045a5efabc6bd35bfcc03f83"
server_key_iv: "1edfa836431daf8a13d091af9d4e8eb2"
server_key_version: "encryptionkey@0"
I20260811 22:23:37.047840 27010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "7acf6ddb739f49a88138f226eea2c915"
format_stamp: "Formatted at 2026-08-11 22:23:37 on dist-test-slave-jq4t"
server_key: "52dbad3e045a5efabc6bd35bfcc03f83"
server_key_iv: "1edfa836431daf8a13d091af9d4e8eb2"
server_key_version: "encryptionkey@0"
I20260811 22:23:37.056463 27010 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.004s
I20260811 22:23:37.061684 27032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:37.063769 27010 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260811 22:23:37.063987 27010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "7acf6ddb739f49a88138f226eea2c915"
format_stamp: "Formatted at 2026-08-11 22:23:37 on dist-test-slave-jq4t"
server_key: "52dbad3e045a5efabc6bd35bfcc03f83"
server_key_iv: "1edfa836431daf8a13d091af9d4e8eb2"
server_key_version: "encryptionkey@0"
I20260811 22:23:37.064216 27010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:37.113685 27010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:37.118512 27010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:37.118854 27010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:37.119938 27010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:37.121490 27010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:37.121598 27010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:37.121757 27010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:37.121833 27010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:37.160676 27010 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:34735
I20260811 22:23:37.160784 27210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:34735 every 8 connection(s)
I20260811 22:23:37.162055 27010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:23:37.169149 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 27010
I20260811 22:23:37.169394 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:23:37.170269 30612 external_mini_cluster.cc:1428] Setting key 78f187142e7074d09641f971d6ea15a9
Aug 11 22:23:37 dist-test-slave-jq4t krb5kdc[26842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487017, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:23:37.187502 27215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:37.164633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:38027 (local address 127.29.229.62:40327)
0811 22:23:37.165136 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:37.165141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:37.165956 (+ 815us) server_negotiation.cc:407] Connection header received
0811 22:23:37.168827 (+ 2871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:37.168836 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:37.168941 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:37.169080 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:37.171384 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:37.172657 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:37.174300 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:37.174940 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:37.175103 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:37.179744 (+ 4641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:37.179753 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:37.179761 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:37.179808 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:37.183531 (+ 3723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:37.184472 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:37.184501 (+ 29us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:37.184507 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:37.184660 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:37.185155 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:37.185163 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:37.185168 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:37.185572 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:37.185719 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:37.186605 (+ 886us) server_negotiation.cc:299] Negotiation successful
0811 22:23:37.186982 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":278,"threads_started":1}
I20260811 22:23:37.188390 27216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:37.165096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:40327 (local address 127.29.229.1:38027)
0811 22:23:37.165794 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:37.165836 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:37.168423 (+ 2587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:37.169376 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:37.169387 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:37.170342 (+ 955us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:37.171153 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:37.171183 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:37.172838 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:37.172843 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:37.174129 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:37.174143 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:37.175207 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:37.175217 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:37.175566 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:37.176322 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:37.176356 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:37.179473 (+ 3117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:37.183687 (+ 4214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:37.183699 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:37.183719 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:37.184337 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:37.184776 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:37.184782 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:37.184788 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:37.185035 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:37.185714 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:37.185724 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:37.186170 (+ 446us) client_negotiation.cc:769] Sending connection context
0811 22:23:37.186464 (+ 294us) client_negotiation.cc:240] Negotiation successful
0811 22:23:37.187024 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":278,"threads_started":1}
I20260811 22:23:37.190287 27212 heartbeater.cc:347] Connected to a master server at 127.29.229.62:40327
I20260811 22:23:37.190618 27212 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:37.191535 27212 heartbeater.cc:511] Master 127.29.229.62:40327 requested a full tablet report, sending...
I20260811 22:23:37.193979 26909 ts_manager.cc:194] Registered new tserver with Master: 7acf6ddb739f49a88138f226eea2c915 (127.29.229.1:34735)
I20260811 22:23:37.195525 26909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:38027
WARNING: no policy specified for kudu/127.29.229.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
I20260811 22:23:37.251866 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:40327
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:37729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:23:37.670153 27219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:37.670617 27219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:37.670944 27219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:37.687742 27219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:37.687944 27219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:37.688210 27219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:23:37.700860 27219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:37729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:40327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:37.702829 27219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:37.704869 27219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:37.720299 27227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:37.720299 27228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:37.720961 27231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:37.722395 27219 server_base.cc:1034] running on GCE node
I20260811 22:23:37.723311 27219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:37.724934 27219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:37.726226 27219 hybrid_clock.cc:648] HybridClock initialized: now 1786487017726133 us; error 99 us; skew 500 ppm
I20260811 22:23:37.726698 27219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:23:37 dist-test-slave-jq4t krb5kdc[26842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487017, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:37.731172 27219 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:23:37.732913 27219 webserver.cc:460] Webserver started at http://127.29.229.2:34741/ using document root <none> and password file <none>
I20260811 22:23:37.733659 27219 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:37.733794 27219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:37.734148 27219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:37.737362 27219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "9c6d1f02762f42a59336ac2d5e62811d"
format_stamp: "Formatted at 2026-08-11 22:23:37 on dist-test-slave-jq4t"
server_key: "7fa0e4bb2b44ba995761245ed1ef9598"
server_key_iv: "39f16482d4bc9fec960460f48a3586d3"
server_key_version: "encryptionkey@0"
I20260811 22:23:37.738195 27219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "9c6d1f02762f42a59336ac2d5e62811d"
format_stamp: "Formatted at 2026-08-11 22:23:37 on dist-test-slave-jq4t"
server_key: "7fa0e4bb2b44ba995761245ed1ef9598"
server_key_iv: "39f16482d4bc9fec960460f48a3586d3"
server_key_version: "encryptionkey@0"
I20260811 22:23:37.743808 27219 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:23:37.747856 27242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:37.749636 27219 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 22:23:37.749858 27219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "9c6d1f02762f42a59336ac2d5e62811d"
format_stamp: "Formatted at 2026-08-11 22:23:37 on dist-test-slave-jq4t"
server_key: "7fa0e4bb2b44ba995761245ed1ef9598"
server_key_iv: "39f16482d4bc9fec960460f48a3586d3"
server_key_version: "encryptionkey@0"
I20260811 22:23:37.750161 27219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:37.773787 27219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:37.778923 27219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:37.779259 27219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:37.780342 27219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:37.781986 27219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:37.782097 27219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:37.782267 27219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:37.782342 27219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:37.827530 27219 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:42381
I20260811 22:23:37.827540 27443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:42381 every 8 connection(s)
I20260811 22:23:37.829589 27219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:23:37.832986 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 27219
I20260811 22:23:37.833189 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:23:37.833841 30612 external_mini_cluster.cc:1428] Setting key 558ace91016e90b37d4b0e74fbc5bfb2
Aug 11 22:23:37 dist-test-slave-jq4t krb5kdc[26842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487017, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:23:37.873154 27447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:37.832799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:36903 (local address 127.29.229.62:40327)
0811 22:23:37.848375 (+ 15576us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:37.848387 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:37.850085 (+ 1698us) server_negotiation.cc:407] Connection header received
0811 22:23:37.854225 (+ 4140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:37.854234 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:37.854353 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:37.854508 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:37.857292 (+ 2784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:37.858152 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:37.860023 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:37.860636 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:37.860869 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:37.866324 (+ 5455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:37.866329 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:37.866333 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:37.866380 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:37.869601 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:37.870630 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:37.870635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:37.870637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:37.870768 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:37.871262 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:37.871274 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:37.871276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:37.871567 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:37.871733 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:37.872525 (+ 792us) server_negotiation.cc:299] Negotiation successful
0811 22:23:37.872761 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15320,"thread_start_us":15605,"threads_started":1}
I20260811 22:23:37.874435 27449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:37.833489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:40327 (local address 127.29.229.2:36903)
0811 22:23:37.849773 (+ 16284us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:37.849857 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:37.853612 (+ 3755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:37.854836 (+ 1224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:37.854853 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:37.855985 (+ 1132us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:37.857024 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:37.857057 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:37.858349 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:37.858353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:37.859858 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:37.859874 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:37.860978 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:37.860997 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:37.861578 (+ 581us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:37.862626 (+ 1048us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:37.862657 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:37.866006 (+ 3349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:37.869765 (+ 3759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:37.869775 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:37.869794 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:37.870491 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:37.870911 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:37.870916 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:37.870920 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:37.871134 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:37.871708 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:37.871717 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:37.872105 (+ 388us) client_negotiation.cc:769] Sending connection context
0811 22:23:37.872401 (+ 296us) client_negotiation.cc:240] Negotiation successful
0811 22:23:37.873030 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15826,"thread_start_us":15246,"threads_started":1}
I20260811 22:23:37.876041 27444 heartbeater.cc:347] Connected to a master server at 127.29.229.62:40327
I20260811 22:23:37.876398 27444 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:37.877185 27444 heartbeater.cc:511] Master 127.29.229.62:40327 requested a full tablet report, sending...
I20260811 22:23:37.878705 26909 ts_manager.cc:194] Registered new tserver with Master: 9c6d1f02762f42a59336ac2d5e62811d (127.29.229.2:42381)
I20260811 22:23:37.879700 26909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:36903
WARNING: no policy specified for kudu/127.29.229.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
I20260811 22:23:37.936822 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:40327
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:37729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:23:38.199465 27212 heartbeater.cc:503] Master 127.29.229.62:40327 was elected leader, sending a full tablet report...
W20260811 22:23:38.302523 27453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:38.302874 27453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:38.303059 27453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:38.310976 27453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:38.311163 27453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:38.311371 27453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:23:38.321729 27453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:37729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:40327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:38.323163 27453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:38.324546 27453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:38.339460 27463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:38.339517 27468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:38.339515 27464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:38.340431 27453 server_base.cc:1034] running on GCE node
I20260811 22:23:38.341133 27453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:38.342147 27453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:38.343415 27453 hybrid_clock.cc:648] HybridClock initialized: now 1786487018343358 us; error 36 us; skew 500 ppm
I20260811 22:23:38.343739 27453 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:23:38 dist-test-slave-jq4t krb5kdc[26842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487018, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:38.347071 27453 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:23:38.348827 27453 webserver.cc:460] Webserver started at http://127.29.229.3:41153/ using document root <none> and password file <none>
I20260811 22:23:38.349653 27453 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:38.349763 27453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:38.350033 27453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:38.353158 27453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "9ab775df5ec74ce59b86cfa2699ded92"
format_stamp: "Formatted at 2026-08-11 22:23:38 on dist-test-slave-jq4t"
server_key: "485eb4e486031c9e7c754ac7921d5567"
server_key_iv: "7e9b427a885adb55e4dcd44a6bb32323"
server_key_version: "encryptionkey@0"
I20260811 22:23:38.354050 27453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "9ab775df5ec74ce59b86cfa2699ded92"
format_stamp: "Formatted at 2026-08-11 22:23:38 on dist-test-slave-jq4t"
server_key: "485eb4e486031c9e7c754ac7921d5567"
server_key_iv: "7e9b427a885adb55e4dcd44a6bb32323"
server_key_version: "encryptionkey@0"
I20260811 22:23:38.359966 27453 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:23:38.364636 27481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:38.366564 27453 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:23:38.366813 27453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "9ab775df5ec74ce59b86cfa2699ded92"
format_stamp: "Formatted at 2026-08-11 22:23:38 on dist-test-slave-jq4t"
server_key: "485eb4e486031c9e7c754ac7921d5567"
server_key_iv: "7e9b427a885adb55e4dcd44a6bb32323"
server_key_version: "encryptionkey@0"
I20260811 22:23:38.367065 27453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:38.390580 27453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:38.396756 27453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:38.397099 27453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:38.398250 27453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:38.399724 27453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:38.399848 27453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:38.400009 27453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:38.400086 27453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:38.439751 27453 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:41023
I20260811 22:23:38.439815 27694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:41023 every 8 connection(s)
I20260811 22:23:38.441227 27453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:23:38.447003 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 27453
I20260811 22:23:38.447206 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:23:38.447690 30612 external_mini_cluster.cc:1428] Setting key 62749eceac2936b4565f60edb8377f4d
Aug 11 22:23:38 dist-test-slave-jq4t krb5kdc[26842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487018, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:23:38.465631 27698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:38.443922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:57673 (local address 127.29.229.62:40327)
0811 22:23:38.444447 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:38.444454 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:38.445407 (+ 953us) server_negotiation.cc:407] Connection header received
0811 22:23:38.447300 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:38.447306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:38.447401 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:38.447529 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:38.449690 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.450670 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.452380 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.452933 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.453077 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:38.457758 (+ 4681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:38.457765 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:38.457769 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:38.457828 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:38.461572 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:38.462618 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:38.462626 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:38.462631 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:38.462805 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:38.463742 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:38.463746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:38.463749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:38.464039 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:38.464167 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:38.465150 (+ 983us) server_negotiation.cc:299] Negotiation successful
0811 22:23:38.465330 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":226,"threads_started":1}
I20260811 22:23:38.466612 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:38.444407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:40327 (local address 127.29.229.3:57673)
0811 22:23:38.445229 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:38.445273 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:38.446981 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:38.447804 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:38.447816 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:38.448760 (+ 944us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:38.449452 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:38.449523 (+ 71us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.450875 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.450880 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:38.452206 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:38.452219 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.453177 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.453184 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:38.453507 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:38.454337 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:38.454363 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:38.457256 (+ 2893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:38.461730 (+ 4474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:38.461740 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:38.461758 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:38.462435 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:38.463053 (+ 618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:38.463061 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:38.463065 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:38.463602 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:38.464204 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:38.464211 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:38.464693 (+ 482us) client_negotiation.cc:769] Sending connection context
0811 22:23:38.464998 (+ 305us) client_negotiation.cc:240] Negotiation successful
0811 22:23:38.465431 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":300,"threads_started":1}
I20260811 22:23:38.467994 27696 heartbeater.cc:347] Connected to a master server at 127.29.229.62:40327
I20260811 22:23:38.468302 27696 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:38.469112 27696 heartbeater.cc:511] Master 127.29.229.62:40327 requested a full tablet report, sending...
I20260811 22:23:38.470618 26909 ts_manager.cc:194] Registered new tserver with Master: 9ab775df5ec74ce59b86cfa2699ded92 (127.29.229.3:41023)
I20260811 22:23:38.471544 26909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:57673
I20260811 22:23:38.476382 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:23:38.495390 27698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:38.481380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38850 (local address 127.29.229.62:40327)
0811 22:23:38.481626 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:38.481632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:38.481931 (+ 299us) server_negotiation.cc:407] Connection header received
0811 22:23:38.482048 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:38.482052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:38.482135 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:38.482223 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:38.483709 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.484536 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.485922 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.486324 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.486424 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:38.489216 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:38.489221 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:38.489225 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:38.489259 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:38.491866 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:38.493091 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:38.493096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:38.493098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:38.493232 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:38.493612 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:38.493616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:38.493618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:38.493983 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:38.494244 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:38.494732 (+ 488us) server_negotiation.cc:299] Negotiation successful
0811 22:23:38.495036 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 22:23:38.500120 26909 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:23:38.507208 26909 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:23:38.539139 27728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:38.522714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44852 (local address 127.29.229.2:42381)
0811 22:23:38.529479 (+ 6765us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:38.529492 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:38.530199 (+ 707us) server_negotiation.cc:407] Connection header received
0811 22:23:38.530456 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:38.530462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:38.530628 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:38.530855 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:38.531605 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.533217 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.536825 (+ 3608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.537895 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.538062 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:38.538297 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:38.538612 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 22:23:38.538794 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6535,"thread_start_us":5710,"threads_started":1}
I20260811 22:23:38.539346 27722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:38.520389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:34735 (local address 127.0.0.1:40674)
0811 22:23:38.521340 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:38.521378 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:38.521570 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:38.529865 (+ 8295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:38.529875 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:38.529940 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:38.530639 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:38.530654 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.533442 (+ 2788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.533449 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:38.536097 (+ 2648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:38.536113 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.538306 (+ 2193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.538314 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:38.538519 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:38.538550 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 22:23:38.538625 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 22:23:38.538791 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":141,"spinlock_wait_cycles":19584,"thread_start_us":9332,"threads_started":1}
I20260811 22:23:38.539968 27726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:38.521227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54544 (local address 127.29.229.3:41023)
0811 22:23:38.529048 (+ 7821us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:38.529060 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:38.529103 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:23:38.529219 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:38.529229 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:38.529529 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:38.530208 (+ 679us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:38.531512 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.533251 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.536815 (+ 3564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.538253 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.538404 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:38.538635 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:38.539459 (+ 824us) server_negotiation.cc:299] Negotiation successful
0811 22:23:38.539628 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7603,"thread_start_us":7447,"threads_started":1}
I20260811 22:23:38.540006 27727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:38.520597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:41023 (local address 127.0.0.1:54544)
0811 22:23:38.522721 (+ 2124us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:38.522750 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:38.522967 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:38.529865 (+ 6898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:38.529875 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:38.529933 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:38.531223 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:38.531239 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.533489 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.533497 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:38.536616 (+ 3119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:38.536632 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.538791 (+ 2159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.538797 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:38.538996 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:38.539028 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:23:38.539123 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 22:23:38.539267 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1896,"mutex_wait_us":281,"spinlock_wait_cycles":768,"thread_start_us":451,"threads_started":1}
I20260811 22:23:38.540236 27721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:38.521829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:42381 (local address 127.0.0.1:44852)
0811 22:23:38.530064 (+ 8235us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:38.530080 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:38.530274 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:38.530860 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:38.530871 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:38.530895 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:38.531399 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:38.531413 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.533375 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.533382 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:38.536616 (+ 3234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:38.536632 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.538130 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.538137 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:38.538335 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:38.538404 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 22:23:38.538553 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 22:23:38.538771 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8108,"mutex_wait_us":151,"thread_start_us":1542,"threads_started":1}
I20260811 22:23:38.540786 27724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:38.520681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40674 (local address 127.29.229.1:34735)
0811 22:23:38.529180 (+ 8499us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:38.529187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:38.529210 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:23:38.529298 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:38.529305 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:38.529525 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:38.531190 (+ 1665us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:38.531525 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.533259 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.536301 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.538036 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.539485 (+ 1449us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:38.539680 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:38.539748 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 22:23:38.539924 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8350,"thread_start_us":8189,"threads_started":1}
I20260811 22:23:38.544034 27321 tablet_service.cc:1467] Processing CreateTablet for tablet 17ca6e0dbbba4f9e92137c32f4b0c2c3 (DEFAULT_TABLE table=test-table [id=cee6de6412e04b8cab74cc034a08f7e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:38.544313 27099 tablet_service.cc:1467] Processing CreateTablet for tablet 17ca6e0dbbba4f9e92137c32f4b0c2c3 (DEFAULT_TABLE table=test-table [id=cee6de6412e04b8cab74cc034a08f7e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:38.544322 27568 tablet_service.cc:1467] Processing CreateTablet for tablet 17ca6e0dbbba4f9e92137c32f4b0c2c3 (DEFAULT_TABLE table=test-table [id=cee6de6412e04b8cab74cc034a08f7e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:38.545506 27321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17ca6e0dbbba4f9e92137c32f4b0c2c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:38.545758 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17ca6e0dbbba4f9e92137c32f4b0c2c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:38.545758 27099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17ca6e0dbbba4f9e92137c32f4b0c2c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:38.558624 27729 tablet_bootstrap.cc:492] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d: Bootstrap starting.
I20260811 22:23:38.560777 27730 tablet_bootstrap.cc:492] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915: Bootstrap starting.
I20260811 22:23:38.563186 27729 tablet_bootstrap.cc:654] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:38.564659 27730 tablet_bootstrap.cc:654] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:38.565498 27735 tablet_bootstrap.cc:492] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92: Bootstrap starting.
I20260811 22:23:38.566479 27730 log.cc:824] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:38.566649 27729 log.cc:824] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:38.568991 27735 tablet_bootstrap.cc:654] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:38.570016 27729 tablet_bootstrap.cc:492] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d: No bootstrap required, opened a new log
I20260811 22:23:38.570036 27730 tablet_bootstrap.cc:492] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915: No bootstrap required, opened a new log
I20260811 22:23:38.570327 27730 ts_tablet_manager.cc:1397] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260811 22:23:38.570327 27729 ts_tablet_manager.cc:1397] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.007s
I20260811 22:23:38.570595 27735 log.cc:824] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:38.573544 27735 tablet_bootstrap.cc:492] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92: No bootstrap required, opened a new log
I20260811 22:23:38.573818 27735 ts_tablet_manager.cc:1397] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 22:23:38.581671 27730 raft_consensus.cc:348] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acf6ddb739f49a88138f226eea2c915" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34735 } } peers { permanent_uuid: "9ab775df5ec74ce59b86cfa2699ded92" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 41023 } } peers { permanent_uuid: "9c6d1f02762f42a59336ac2d5e62811d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42381 } }
I20260811 22:23:38.581670 27729 raft_consensus.cc:348] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acf6ddb739f49a88138f226eea2c915" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34735 } } peers { permanent_uuid: "9ab775df5ec74ce59b86cfa2699ded92" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 41023 } } peers { permanent_uuid: "9c6d1f02762f42a59336ac2d5e62811d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42381 } }
I20260811 22:23:38.582193 27729 raft_consensus.cc:374] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:38.582183 27730 raft_consensus.cc:374] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:38.581959 27735 raft_consensus.cc:348] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acf6ddb739f49a88138f226eea2c915" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34735 } } peers { permanent_uuid: "9ab775df5ec74ce59b86cfa2699ded92" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 41023 } } peers { permanent_uuid: "9c6d1f02762f42a59336ac2d5e62811d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42381 } }
I20260811 22:23:38.582297 27729 raft_consensus.cc:729] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c6d1f02762f42a59336ac2d5e62811d, State: Initialized, Role: FOLLOWER
I20260811 22:23:38.582296 27730 raft_consensus.cc:729] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7acf6ddb739f49a88138f226eea2c915, State: Initialized, Role: FOLLOWER
I20260811 22:23:38.582412 27735 raft_consensus.cc:374] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:38.582530 27735 raft_consensus.cc:729] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ab775df5ec74ce59b86cfa2699ded92, State: Initialized, Role: FOLLOWER
I20260811 22:23:38.583004 27729 consensus_queue.cc:260] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acf6ddb739f49a88138f226eea2c915" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34735 } } peers { permanent_uuid: "9ab775df5ec74ce59b86cfa2699ded92" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 41023 } } peers { permanent_uuid: "9c6d1f02762f42a59336ac2d5e62811d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42381 } }
I20260811 22:23:38.582966 27730 consensus_queue.cc:260] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acf6ddb739f49a88138f226eea2c915" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34735 } } peers { permanent_uuid: "9ab775df5ec74ce59b86cfa2699ded92" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 41023 } } peers { permanent_uuid: "9c6d1f02762f42a59336ac2d5e62811d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42381 } }
I20260811 22:23:38.583119 27735 consensus_queue.cc:260] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acf6ddb739f49a88138f226eea2c915" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34735 } } peers { permanent_uuid: "9ab775df5ec74ce59b86cfa2699ded92" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 41023 } } peers { permanent_uuid: "9c6d1f02762f42a59336ac2d5e62811d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42381 } }
I20260811 22:23:38.584592 27444 heartbeater.cc:503] Master 127.29.229.62:40327 was elected leader, sending a full tablet report...
I20260811 22:23:38.585868 27730 ts_tablet_manager.cc:1428] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915: Time spent starting tablet: real 0.015s user 0.012s sys 0.003s
I20260811 22:23:38.587037 27729 ts_tablet_manager.cc:1428] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d: Time spent starting tablet: real 0.017s user 0.013s sys 0.002s
I20260811 22:23:38.588500 27696 heartbeater.cc:503] Master 127.29.229.62:40327 was elected leader, sending a full tablet report...
I20260811 22:23:38.590775 27735 ts_tablet_manager.cc:1428] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92: Time spent starting tablet: real 0.017s user 0.011s sys 0.005s
W20260811 22:23:38.664978 27214 tablet.cc:2369] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:23:38.693010 27697 tablet.cc:2369] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:23:38.751168 27743 raft_consensus.cc:482] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:23:38.751534 27743 raft_consensus.cc:504] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acf6ddb739f49a88138f226eea2c915" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34735 } } peers { permanent_uuid: "9ab775df5ec74ce59b86cfa2699ded92" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 41023 } } peers { permanent_uuid: "9c6d1f02762f42a59336ac2d5e62811d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42381 } }
I20260811 22:23:38.753613 27743 leader_election.cc:288] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7acf6ddb739f49a88138f226eea2c915 (127.29.229.1:34735), 9c6d1f02762f42a59336ac2d5e62811d (127.29.229.2:42381)
I20260811 22:23:38.760392 27724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:38.754220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:33621 (local address 127.29.229.1:34735)
0811 22:23:38.754481 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:38.754489 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:38.754535 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:23:38.754662 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:38.754668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:38.754758 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:38.754856 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:38.755405 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.756423 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.758326 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.759446 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.759570 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:38.759623 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:38.759985 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 22:23:38.760080 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260811 22:23:38.760411 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:38.753936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:34735 (local address 127.29.229.3:33621)
0811 22:23:38.754289 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:38.754312 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:38.754472 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:38.754904 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:38.754908 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:38.754932 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:38.755218 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:38.755229 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.756601 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.756607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:38.758151 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:38.758164 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.759600 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.759607 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:38.759805 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:38.759844 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 22:23:38.759953 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 22:23:38.760094 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"spinlock_wait_cycles":56064}
I20260811 22:23:38.761453 27141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17ca6e0dbbba4f9e92137c32f4b0c2c3" candidate_uuid: "9ab775df5ec74ce59b86cfa2699ded92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7acf6ddb739f49a88138f226eea2c915" is_pre_election: true
I20260811 22:23:38.761679 27728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:38.754239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:38533 (local address 127.29.229.2:42381)
0811 22:23:38.754759 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:38.754766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:38.755020 (+ 254us) server_negotiation.cc:407] Connection header received
0811 22:23:38.755216 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:38.755221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:38.755307 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:38.755436 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:38.756124 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.757128 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.759417 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.760739 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.760933 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:38.761008 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:38.761278 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 22:23:38.761410 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367}
I20260811 22:23:38.761792 27746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:38.753994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:42381 (local address 127.29.229.3:38533)
0811 22:23:38.754900 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:38.754919 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:38.755074 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:38.755524 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:38.755528 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:38.755554 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:38.755945 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:38.755957 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.757291 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.757295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:38.759027 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:38.759043 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.760935 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.760941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:38.761114 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:38.761137 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:23:38.761247 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 22:23:38.761400 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"spinlock_wait_cycles":1920,"thread_start_us":551,"threads_started":1}
I20260811 22:23:38.762084 27141 raft_consensus.cc:2454] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ab775df5ec74ce59b86cfa2699ded92 in term 0.
I20260811 22:23:38.762810 27359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17ca6e0dbbba4f9e92137c32f4b0c2c3" candidate_uuid: "9ab775df5ec74ce59b86cfa2699ded92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c6d1f02762f42a59336ac2d5e62811d" is_pre_election: true
I20260811 22:23:38.763083 27483 leader_election.cc:302] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7acf6ddb739f49a88138f226eea2c915, 9ab775df5ec74ce59b86cfa2699ded92; no voters:
I20260811 22:23:38.763407 27359 raft_consensus.cc:2454] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ab775df5ec74ce59b86cfa2699ded92 in term 0.
I20260811 22:23:38.763777 27743 raft_consensus.cc:2781] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:23:38.763974 27743 raft_consensus.cc:482] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:23:38.764117 27743 raft_consensus.cc:3037] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:38.766748 27743 raft_consensus.cc:504] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acf6ddb739f49a88138f226eea2c915" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34735 } } peers { permanent_uuid: "9ab775df5ec74ce59b86cfa2699ded92" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 41023 } } peers { permanent_uuid: "9c6d1f02762f42a59336ac2d5e62811d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42381 } }
I20260811 22:23:38.767410 27743 leader_election.cc:288] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [CANDIDATE]: Term 1 election: Requested vote from peers 7acf6ddb739f49a88138f226eea2c915 (127.29.229.1:34735), 9c6d1f02762f42a59336ac2d5e62811d (127.29.229.2:42381)
I20260811 22:23:38.768014 27359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17ca6e0dbbba4f9e92137c32f4b0c2c3" candidate_uuid: "9ab775df5ec74ce59b86cfa2699ded92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c6d1f02762f42a59336ac2d5e62811d"
I20260811 22:23:38.768024 27141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17ca6e0dbbba4f9e92137c32f4b0c2c3" candidate_uuid: "9ab775df5ec74ce59b86cfa2699ded92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7acf6ddb739f49a88138f226eea2c915"
I20260811 22:23:38.768267 27359 raft_consensus.cc:3037] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:38.768267 27141 raft_consensus.cc:3037] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:38.771839 27359 raft_consensus.cc:2454] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ab775df5ec74ce59b86cfa2699ded92 in term 1.
I20260811 22:23:38.771839 27141 raft_consensus.cc:2454] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ab775df5ec74ce59b86cfa2699ded92 in term 1.
I20260811 22:23:38.772374 27484 leader_election.cc:302] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ab775df5ec74ce59b86cfa2699ded92, 9c6d1f02762f42a59336ac2d5e62811d; no voters:
I20260811 22:23:38.772805 27743 raft_consensus.cc:2781] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:23:38.773277 27743 raft_consensus.cc:686] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [term 1 LEADER]: Becoming Leader. State: Replica: 9ab775df5ec74ce59b86cfa2699ded92, State: Running, Role: LEADER
I20260811 22:23:38.773797 27743 consensus_queue.cc:237] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acf6ddb739f49a88138f226eea2c915" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34735 } } peers { permanent_uuid: "9ab775df5ec74ce59b86cfa2699ded92" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 41023 } } peers { permanent_uuid: "9c6d1f02762f42a59336ac2d5e62811d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42381 } }
I20260811 22:23:38.779050 26906 catalog_manager.cc:5495] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ab775df5ec74ce59b86cfa2699ded92 (127.29.229.3). New cstate: current_term: 1 leader_uuid: "9ab775df5ec74ce59b86cfa2699ded92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acf6ddb739f49a88138f226eea2c915" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ab775df5ec74ce59b86cfa2699ded92" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 41023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c6d1f02762f42a59336ac2d5e62811d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42381 } health_report { overall_health: UNKNOWN } } }
W20260811 22:23:38.832312 27446 tablet.cc:2369] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:23:38.835960 27726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:38.830032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54552 (local address 127.29.229.3:41023)
0811 22:23:38.830236 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:38.830241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:38.830260 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:23:38.830473 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:38.830478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:38.830570 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:38.830725 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:23:38.831129 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.831872 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.833725 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.834145 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.834290 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:38.834835 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:23:38.835320 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:23:38.835482 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:38.835600 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 22:23:38.835690 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:23:38.845352 27359 raft_consensus.cc:1261] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d [term 1 FOLLOWER]: Refusing update from remote peer 9ab775df5ec74ce59b86cfa2699ded92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:23:38.845366 27141 raft_consensus.cc:1261] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915 [term 1 FOLLOWER]: Refusing update from remote peer 9ab775df5ec74ce59b86cfa2699ded92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:23:38.846297 27747 consensus_queue.cc:1035] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7acf6ddb739f49a88138f226eea2c915" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 34735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:23:38.846680 27743 consensus_queue.cc:1035] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c6d1f02762f42a59336ac2d5e62811d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:23:38.858975 27754 mvcc.cc:205] Tried to move back new op lower bound from 7317450829179289600 to 7317450828901535744. Current Snapshot: MvccSnapshot[applied={T|T < 7317450829179289600}]
I20260811 22:23:38.872357 26909 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:23:38.872807 26909 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:23:38.876700 26909 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d761263c466249b1887fded9096b5379: Sending DeleteTablet for 3 replicas of tablet 17ca6e0dbbba4f9e92137c32f4b0c2c3
I20260811 22:23:38.878014 27568 tablet_service.cc:1514] Processing DeleteTablet for tablet 17ca6e0dbbba4f9e92137c32f4b0c2c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:38 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:54544
I20260811 22:23:38.878125 27099 tablet_service.cc:1514] Processing DeleteTablet for tablet 17ca6e0dbbba4f9e92137c32f4b0c2c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:38 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:40674
I20260811 22:23:38.878412 27321 tablet_service.cc:1514] Processing DeleteTablet for tablet 17ca6e0dbbba4f9e92137c32f4b0c2c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:38 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:44852
I20260811 22:23:38.879611 27776 tablet_replica.cc:330] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92: stopping tablet replica
I20260811 22:23:38.880050 27776 raft_consensus.cc:2229] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:23:38.880810 27776 raft_consensus.cc:2258] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:38.882634 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 27010
I20260811 22:23:38.883566 27776 ts_tablet_manager.cc:1905] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:38.895684 27777 tablet_replica.cc:330] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915: stopping tablet replica
I20260811 22:23:38.895879 27776 ts_tablet_manager.cc:1918] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:23:38.896106 27776 log.cc:1196] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/17ca6e0dbbba4f9e92137c32f4b0c2c3
I20260811 22:23:38.896697 27777 raft_consensus.cc:2229] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:23:38.896777 27776 ts_tablet_manager.cc:1939] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9ab775df5ec74ce59b86cfa2699ded92: Deleting consensus metadata
I20260811 22:23:38.897328 27777 raft_consensus.cc:2258] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:38.898208 27783 tablet_replica.cc:330] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d: stopping tablet replica
I20260811 22:23:38.898663 27783 raft_consensus.cc:2229] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:23:38.899257 27783 raft_consensus.cc:2258] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:38.900045 27777 ts_tablet_manager.cc:1905] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:38.901544 27783 ts_tablet_manager.cc:1905] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:38.902242 26886 catalog_manager.cc:4841] TS 9ab775df5ec74ce59b86cfa2699ded92 (127.29.229.3:41023): tablet 17ca6e0dbbba4f9e92137c32f4b0c2c3 (table test-table [id=cee6de6412e04b8cab74cc034a08f7e8]) successfully deleted
I20260811 22:23:38.907230 27777 ts_tablet_manager.cc:1918] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:23:38.907440 27777 log.cc:1196] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-0/wal/wals/17ca6e0dbbba4f9e92137c32f4b0c2c3
I20260811 22:23:38.907985 27777 ts_tablet_manager.cc:1939] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 7acf6ddb739f49a88138f226eea2c915: Deleting consensus metadata
I20260811 22:23:38.909292 27783 ts_tablet_manager.cc:1918] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:23:38.909487 27783 log.cc:1196] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/17ca6e0dbbba4f9e92137c32f4b0c2c3
I20260811 22:23:38.909961 27783 ts_tablet_manager.cc:1939] T 17ca6e0dbbba4f9e92137c32f4b0c2c3 P 9c6d1f02762f42a59336ac2d5e62811d: Deleting consensus metadata
Aug 11 22:23:38 dist-test-slave-jq4t krb5kdc[26842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:23:38.914179 26885 catalog_manager.cc:4841] TS 7acf6ddb739f49a88138f226eea2c915 (127.29.229.1:34735): tablet 17ca6e0dbbba4f9e92137c32f4b0c2c3 (table test-table [id=cee6de6412e04b8cab74cc034a08f7e8]) successfully deleted
I20260811 22:23:38.916047 26886 catalog_manager.cc:4841] TS 9c6d1f02762f42a59336ac2d5e62811d (127.29.229.2:42381): tablet 17ca6e0dbbba4f9e92137c32f4b0c2c3 (table test-table [id=cee6de6412e04b8cab74cc034a08f7e8]) successfully deleted
I20260811 22:23:38.920614 27724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:38.884134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40676 (local address 127.29.229.1:34735)
0811 22:23:38.884334 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:38.884339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:38.893389 (+ 9050us) server_negotiation.cc:407] Connection header received
0811 22:23:38.893627 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:38.893632 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:38.893751 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:38.893853 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:38.895834 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.897105 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.899250 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:38.899932 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:38.900043 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:38.912274 (+ 12231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:38.912290 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:38.912311 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:38.912366 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:38.917017 (+ 4651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:38.918159 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:38.918169 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:38.918173 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:38.918361 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:38.918844 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:38.918850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:38.918853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:38.919317 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:38.919643 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:38.919970 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 22:23:38.920200 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:23:38.921500 27205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:39.047534 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 27010
I20260811 22:23:39.076917 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 27219
Aug 11 22:23:39 dist-test-slave-jq4t krb5kdc[26842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.2@KRBTEST.COM
I20260811 22:23:39.096277 27728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:39.077813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44858 (local address 127.29.229.2:42381)
0811 22:23:39.078033 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:39.078041 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:39.078065 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:23:39.078139 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:39.078145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:39.078242 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:39.078357 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:39.079631 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:39.080833 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:39.082676 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:39.083356 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:39.083514 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:39.088344 (+ 4830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:39.088380 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:39.088412 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:39.088539 (+ 127us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:39.093089 (+ 4550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:39.093917 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:39.093929 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:39.093933 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:39.094108 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:39.094534 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:39.094540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:39.094543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:39.095033 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:39.095355 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:39.095628 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 22:23:39.095872 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:23:39.097093 27437 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:39.193303 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 27219
I20260811 22:23:39.221962 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 27453
Aug 11 22:23:39 dist-test-slave-jq4t krb5kdc[26842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.3@KRBTEST.COM
I20260811 22:23:39.239279 27726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:39.222787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54564 (local address 127.29.229.3:41023)
0811 22:23:39.222976 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:39.222983 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:39.223006 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:23:39.223226 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:39.223231 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:39.223331 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:39.223461 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:39.224901 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:39.225727 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:39.227132 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:39.227601 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:39.227747 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:39.232466 (+ 4719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:39.232524 (+ 58us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:39.232543 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:39.232597 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:39.235846 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:39.236728 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:39.236736 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:39.236738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:39.237048 (+ 310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:39.237518 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:39.237523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:39.237527 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:39.237948 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:39.238338 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:39.238627 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 22:23:39.238887 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:23:39.240094 27689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:39.370103 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 27453
I20260811 22:23:39.402968 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 26865
I20260811 22:23:39.404206 26959 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:39.517951 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 26865
2026-08-11T22:23:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5438 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T22:23:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:23:39Z Disabled control of system clock
I20260811 22:23:39.627784 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:42897
--webserver_interface=127.29.229.62
--webserver_port=0
--builtin_ntp_servers=127.29.229.20:34047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:42897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:23:40.004078 27795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:40.004415 27795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:40.004567 27795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:40.012215 27795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:23:40.012363 27795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:40.012437 27795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:40.012549 27795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:23:40.012632 27795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:23:40.022485 27795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:34047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:42897
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:42897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:40.023957 27795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:40.025915 27795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:40.038988 27806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:40.038966 27809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:40.039031 27805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:40.042536 27795 server_base.cc:1034] running on GCE node
I20260811 22:23:40.043744 27795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:40.045518 27795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:40.046842 27795 hybrid_clock.cc:648] HybridClock initialized: now 1786487020046749 us; error 87 us; skew 500 ppm
I20260811 22:23:40.047405 27795 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:23:40.050724 27795 webserver.cc:460] Webserver started at http://127.29.229.62:46815/ using document root <none> and password file <none>
I20260811 22:23:40.051676 27795 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:40.051827 27795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:40.052194 27795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:40.055552 27795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "f2bca6d435644c46b3bc018c65ec2ed7"
format_stamp: "Formatted at 2026-08-11 22:23:40 on dist-test-slave-jq4t"
server_key: "39222d668c88403632b1b0cc4c61d6e5"
server_key_iv: "4f0af3cf976bbe5b2c52e8bcb92e5301"
server_key_version: "encryptionkey@0"
I20260811 22:23:40.056466 27795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "f2bca6d435644c46b3bc018c65ec2ed7"
format_stamp: "Formatted at 2026-08-11 22:23:40 on dist-test-slave-jq4t"
server_key: "39222d668c88403632b1b0cc4c61d6e5"
server_key_iv: "4f0af3cf976bbe5b2c52e8bcb92e5301"
server_key_version: "encryptionkey@0"
I20260811 22:23:40.062095 27795 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 22:23:40.066404 27823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:40.067888 27795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:23:40.068136 27795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "f2bca6d435644c46b3bc018c65ec2ed7"
format_stamp: "Formatted at 2026-08-11 22:23:40 on dist-test-slave-jq4t"
server_key: "39222d668c88403632b1b0cc4c61d6e5"
server_key_iv: "4f0af3cf976bbe5b2c52e8bcb92e5301"
server_key_version: "encryptionkey@0"
I20260811 22:23:40.068400 27795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:40.092181 27795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:40.093217 27795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:40.093532 27795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:40.114025 27795 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:42897
I20260811 22:23:40.114048 27904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:42897 every 8 connection(s)
I20260811 22:23:40.115546 27795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:23:40.119783 27905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:40.125111 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 27795
I20260811 22:23:40.125334 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:23:40.125789 30612 external_mini_cluster.cc:1428] Setting key 1308074ca6a26a1c189b9ae6664bfccf
I20260811 22:23:40.131314 27905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7: Bootstrap starting.
I20260811 22:23:40.135355 27905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:40.136869 27905 log.cc:824] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:40.140527 27905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7: No bootstrap required, opened a new log
I20260811 22:23:40.142007 27909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:40.127921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40256 (local address 127.29.229.62:42897)
0811 22:23:40.129065 (+ 1144us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:40.129079 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:40.129124 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:23:40.130990 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:40.131011 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:40.131455 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:40.131843 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:40.132602 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:40.133997 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:40.136225 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:40.137100 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:40.137242 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:40.138629 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:40.138639 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:40.138658 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:40.139386 (+ 728us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:40.139515 (+ 129us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:40.139547 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:40.139667 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:40.139966 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 22:23:40.140301 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":482,"threads_started":1}
I20260811 22:23:40.152313 27905 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2bca6d435644c46b3bc018c65ec2ed7" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42897 } }
I20260811 22:23:40.152858 27905 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:40.152985 27905 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2bca6d435644c46b3bc018c65ec2ed7, State: Initialized, Role: FOLLOWER
I20260811 22:23:40.153716 27905 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2bca6d435644c46b3bc018c65ec2ed7" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42897 } }
I20260811 22:23:40.154083 27905 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:23:40.154281 27905 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:23:40.154491 27905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:40.156932 27905 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2bca6d435644c46b3bc018c65ec2ed7" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42897 } }
I20260811 22:23:40.157620 27905 leader_election.cc:302] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2bca6d435644c46b3bc018c65ec2ed7; no voters:
I20260811 22:23:40.158695 27905 leader_election.cc:288] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:23:40.158931 27913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:23:40.160073 27913 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7 [term 1 LEADER]: Becoming Leader. State: Replica: f2bca6d435644c46b3bc018c65ec2ed7, State: Running, Role: LEADER
I20260811 22:23:40.160763 27913 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2bca6d435644c46b3bc018c65ec2ed7" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42897 } }
I20260811 22:23:40.161046 27905 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:23:40.164633 27916 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2bca6d435644c46b3bc018c65ec2ed7. Latest consensus state: current_term: 1 leader_uuid: "f2bca6d435644c46b3bc018c65ec2ed7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2bca6d435644c46b3bc018c65ec2ed7" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42897 } } }
I20260811 22:23:40.165606 27916 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7 [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:40.165220 27914 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2bca6d435644c46b3bc018c65ec2ed7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2bca6d435644c46b3bc018c65ec2ed7" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 42897 } } }
I20260811 22:23:40.165863 27914 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7 [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:40.168793 27929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:23:40.174268 27929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:23:40.183367 27929 catalog_manager.cc:1347] Generated new cluster ID: bb8e72f77b394d7dbffc00e0326776ea
I20260811 22:23:40.183522 27929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:23:40.198225 27929 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:23:40.200235 27929 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:23:40.217953 27929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7: Generated new TSK 0
I20260811 22:23:40.219121 27929 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:23:40.237495 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:42897
--builtin_ntp_servers=127.29.229.20:34047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:23:40.620725 27945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:40.621037 27945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:40.621135 27945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:40.628545 27945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:40.628652 27945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:40.628759 27945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:23:40.638126 27945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:34047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:42897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:40.639534 27945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:40.641410 27945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:40.655290 27952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:40.655478 27957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:40.655696 27951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:40.656880 27945 server_base.cc:1034] running on GCE node
I20260811 22:23:40.657574 27945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:40.658658 27945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:40.659922 27945 hybrid_clock.cc:648] HybridClock initialized: now 1786487020659862 us; error 48 us; skew 500 ppm
I20260811 22:23:40.660238 27945 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:23:40.662858 27945 webserver.cc:460] Webserver started at http://127.29.229.1:46085/ using document root <none> and password file <none>
I20260811 22:23:40.663717 27945 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:40.663810 27945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:40.664070 27945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:40.667218 27945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "1dba52d467624ffc80a62ef0bbf66bc2"
format_stamp: "Formatted at 2026-08-11 22:23:40 on dist-test-slave-jq4t"
server_key: "635b9f93c259051c11e3a81417b58c87"
server_key_iv: "dee2d2cdd1df665c36adc6f6070a4c3b"
server_key_version: "encryptionkey@0"
I20260811 22:23:40.667999 27945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "1dba52d467624ffc80a62ef0bbf66bc2"
format_stamp: "Formatted at 2026-08-11 22:23:40 on dist-test-slave-jq4t"
server_key: "635b9f93c259051c11e3a81417b58c87"
server_key_iv: "dee2d2cdd1df665c36adc6f6070a4c3b"
server_key_version: "encryptionkey@0"
I20260811 22:23:40.673522 27945 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 22:23:40.677349 27969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:40.678755 27945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:23:40.678902 27945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "1dba52d467624ffc80a62ef0bbf66bc2"
format_stamp: "Formatted at 2026-08-11 22:23:40 on dist-test-slave-jq4t"
server_key: "635b9f93c259051c11e3a81417b58c87"
server_key_iv: "dee2d2cdd1df665c36adc6f6070a4c3b"
server_key_version: "encryptionkey@0"
I20260811 22:23:40.679091 27945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:40.702317 27945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:40.703222 27945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:40.703509 27945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:40.704648 27945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:40.706387 27945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:40.706496 27945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:40.706578 27945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:40.706635 27945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:40.747221 28161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:41049 every 8 connection(s)
I20260811 22:23:40.747221 27945 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:41049
I20260811 22:23:40.749022 27945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:23:40.757925 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 27945
I20260811 22:23:40.758157 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:23:40.758662 30612 external_mini_cluster.cc:1428] Setting key 4971b5b9e8732f363bc9823e3d9fa6ad
I20260811 22:23:40.761780 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:42897
--builtin_ntp_servers=127.29.229.20:34047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:23:40.762825 28164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:40.751820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:56189 (local address 127.29.229.62:42897)
0811 22:23:40.752364 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:40.752369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:40.753703 (+ 1334us) server_negotiation.cc:407] Connection header received
0811 22:23:40.755595 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:40.755602 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:40.755705 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:40.755835 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:40.757000 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:40.757797 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:40.759635 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:40.760064 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:40.760212 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:40.761829 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:40.761834 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:40.761837 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:40.761893 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:40.761951 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:40.761986 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:40.762077 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:40.762317 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 22:23:40.762450 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":293,"threads_started":1}
I20260811 22:23:40.764155 28165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:40.752442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:42897 (local address 127.29.229.1:56189)
0811 22:23:40.753505 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:40.753551 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:40.755298 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:40.756093 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:40.756102 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:40.756191 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:40.756817 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:40.756834 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:40.757976 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:40.757986 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:40.759477 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:40.759492 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:40.760321 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:40.760330 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:40.760624 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:40.761479 (+ 855us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:23:40.761484 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:40.761534 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:23:40.761622 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:40.762077 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:40.762092 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:40.762102 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 22:23:40.762289 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 22:23:40.762710 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":318,"threads_started":1}
I20260811 22:23:40.766275 28162 heartbeater.cc:347] Connected to a master server at 127.29.229.62:42897
I20260811 22:23:40.766624 28162 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:40.767486 28162 heartbeater.cc:511] Master 127.29.229.62:42897 requested a full tablet report, sending...
I20260811 22:23:40.770365 27844 ts_manager.cc:194] Registered new tserver with Master: 1dba52d467624ffc80a62ef0bbf66bc2 (127.29.229.1:41049)
I20260811 22:23:40.771968 27844 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.229.1:56189
W20260811 22:23:41.129257 28172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:41.129554 28172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:41.129644 28172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:41.137181 28172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:41.137298 28172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:41.137411 28172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:23:41.146643 28172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:34047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:42897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:41.148005 28172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:41.149807 28172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:41.164815 28179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:41.164765 28183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:41.164934 28181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:41.166396 28172 server_base.cc:1034] running on GCE node
I20260811 22:23:41.167155 28172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:41.168236 28172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:41.169533 28172 hybrid_clock.cc:648] HybridClock initialized: now 1786487021169450 us; error 98 us; skew 500 ppm
I20260811 22:23:41.169899 28172 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:23:41.172643 28172 webserver.cc:460] Webserver started at http://127.29.229.2:41431/ using document root <none> and password file <none>
I20260811 22:23:41.173403 28172 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:41.173550 28172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:41.173908 28172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:41.177050 28172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "f9d12a4ab1fa443cb5a915894207d914"
format_stamp: "Formatted at 2026-08-11 22:23:41 on dist-test-slave-jq4t"
server_key: "c34f2022ce617d958d8ee8b891c119a2"
server_key_iv: "30e13e4b1b6af9ee6031796da62eddd9"
server_key_version: "encryptionkey@0"
I20260811 22:23:41.177937 28172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "f9d12a4ab1fa443cb5a915894207d914"
format_stamp: "Formatted at 2026-08-11 22:23:41 on dist-test-slave-jq4t"
server_key: "c34f2022ce617d958d8ee8b891c119a2"
server_key_iv: "30e13e4b1b6af9ee6031796da62eddd9"
server_key_version: "encryptionkey@0"
I20260811 22:23:41.184340 28172 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 22:23:41.189386 28190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:41.191308 28172 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.004s
I20260811 22:23:41.191535 28172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "f9d12a4ab1fa443cb5a915894207d914"
format_stamp: "Formatted at 2026-08-11 22:23:41 on dist-test-slave-jq4t"
server_key: "c34f2022ce617d958d8ee8b891c119a2"
server_key_iv: "30e13e4b1b6af9ee6031796da62eddd9"
server_key_version: "encryptionkey@0"
I20260811 22:23:41.191794 28172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:41.221596 28172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:41.222524 28172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:41.222857 28172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:41.223909 28172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:41.225687 28172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:41.225827 28172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:41.225979 28172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:41.226074 28172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:41.265874 28172 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:40967
I20260811 22:23:41.265970 28381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:40967 every 8 connection(s)
I20260811 22:23:41.267345 28172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:23:41.271589 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 28172
I20260811 22:23:41.271826 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:23:41.272459 30612 external_mini_cluster.cc:1428] Setting key e9650a08e44b57bfa7a4c292bbeb3388
I20260811 22:23:41.276041 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:42897
--builtin_ntp_servers=127.29.229.20:34047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:23:41.283407 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:41.270467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:36049 (local address 127.29.229.62:42897)
0811 22:23:41.271048 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:41.271058 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:41.273367 (+ 2309us) server_negotiation.cc:407] Connection header received
0811 22:23:41.274769 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:41.274775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:41.274871 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:41.274996 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:41.276524 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.277274 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.279888 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.280335 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.280446 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:41.282479 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:41.282484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:41.282487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:41.282508 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:41.282550 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:41.282563 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:41.282641 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:41.282932 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 22:23:41.283087 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":278,"threads_started":1}
I20260811 22:23:41.284911 28391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:41.271102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:42897 (local address 127.29.229.2:36049)
0811 22:23:41.272391 (+ 1289us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:41.272439 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:41.274492 (+ 2053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:41.275276 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:41.275288 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:41.275407 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:41.276271 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:41.276309 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.277447 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.277456 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:41.279712 (+ 2256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:41.279730 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.280621 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.280632 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:41.280980 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:41.282049 (+ 1069us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:23:41.282055 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:41.282121 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:23:41.282231 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:41.282654 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:41.282667 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:41.282679 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 22:23:41.282891 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 22:23:41.283341 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":471,"threads_started":1}
I20260811 22:23:41.286541 28382 heartbeater.cc:347] Connected to a master server at 127.29.229.62:42897
I20260811 22:23:41.286893 28382 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:41.287792 28382 heartbeater.cc:511] Master 127.29.229.62:42897 requested a full tablet report, sending...
I20260811 22:23:41.289418 27844 ts_manager.cc:194] Registered new tserver with Master: f9d12a4ab1fa443cb5a915894207d914 (127.29.229.2:40967)
I20260811 22:23:41.290256 27844 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.229.2:36049
W20260811 22:23:41.650297 28392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:41.650614 28392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:41.650776 28392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:41.658349 28392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:41.658526 28392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:41.658713 28392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:23:41.668530 28392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:34047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:42897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:41.669960 28392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:41.671960 28392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:41.688591 28400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:41.688453 28401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:41.688453 28406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:41.689705 28392 server_base.cc:1034] running on GCE node
I20260811 22:23:41.690366 28392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:41.691500 28392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:41.692796 28392 hybrid_clock.cc:648] HybridClock initialized: now 1786487021692748 us; error 57 us; skew 500 ppm
I20260811 22:23:41.693204 28392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:23:41.696045 28392 webserver.cc:460] Webserver started at http://127.29.229.3:38245/ using document root <none> and password file <none>
I20260811 22:23:41.696913 28392 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:41.697037 28392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:41.697374 28392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:41.700464 28392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "93021a8417ec42c4a68f7f836216b8ac"
format_stamp: "Formatted at 2026-08-11 22:23:41 on dist-test-slave-jq4t"
server_key: "994c010d881c75207450ff1f2a53b37a"
server_key_iv: "c27cff6ef0404b9a66a373d9ab3e89a3"
server_key_version: "encryptionkey@0"
I20260811 22:23:41.701418 28392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "93021a8417ec42c4a68f7f836216b8ac"
format_stamp: "Formatted at 2026-08-11 22:23:41 on dist-test-slave-jq4t"
server_key: "994c010d881c75207450ff1f2a53b37a"
server_key_iv: "c27cff6ef0404b9a66a373d9ab3e89a3"
server_key_version: "encryptionkey@0"
I20260811 22:23:41.707765 28392 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260811 22:23:41.711941 28418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:41.713821 28392 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:23:41.714069 28392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "93021a8417ec42c4a68f7f836216b8ac"
format_stamp: "Formatted at 2026-08-11 22:23:41 on dist-test-slave-jq4t"
server_key: "994c010d881c75207450ff1f2a53b37a"
server_key_iv: "c27cff6ef0404b9a66a373d9ab3e89a3"
server_key_version: "encryptionkey@0"
I20260811 22:23:41.714339 28392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:41.737650 28392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:41.738715 28392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:41.739068 28392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:41.740212 28392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:41.741940 28392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:41.742069 28392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:41.742218 28392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:41.742316 28392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:41.775197 28162 heartbeater.cc:503] Master 127.29.229.62:42897 was elected leader, sending a full tablet report...
I20260811 22:23:41.785477 28392 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:43279
I20260811 22:23:41.785588 28598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:43279 every 8 connection(s)
I20260811 22:23:41.786881 28392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:23:41.788297 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 28392
I20260811 22:23:41.788537 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:23:41.789120 30612 external_mini_cluster.cc:1428] Setting key b3662b27a2365f0a5e7ad53500799950
I20260811 22:23:41.800917 28603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:41.790006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:44951 (local address 127.29.229.62:42897)
0811 22:23:41.790805 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:41.790812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:41.792099 (+ 1287us) server_negotiation.cc:407] Connection header received
0811 22:23:41.794093 (+ 1994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:41.794098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:41.794172 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:41.794279 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:41.795437 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.796292 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.798010 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.798397 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.798547 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:41.799986 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:41.799991 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:41.799994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:41.800014 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:41.800055 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:41.800067 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:41.800169 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:41.800401 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 22:23:41.800536 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":387,"threads_started":1}
I20260811 22:23:41.801927 28604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:41.790791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:42897 (local address 127.29.229.3:44951)
0811 22:23:41.791939 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:41.791985 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:41.793794 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:41.794491 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:41.794499 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:41.794585 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:41.795225 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:41.795243 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.796562 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.796566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:41.797828 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:41.797842 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.798646 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.798654 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:41.798896 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:41.799642 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:23:41.799645 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:41.799685 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:23:41.799770 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:41.800171 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:41.800186 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:41.800196 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 22:23:41.800391 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 22:23:41.800844 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":360,"threads_started":1}
I20260811 22:23:41.804059 28599 heartbeater.cc:347] Connected to a master server at 127.29.229.62:42897
I20260811 22:23:41.804404 28599 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:41.805334 28599 heartbeater.cc:511] Master 127.29.229.62:42897 requested a full tablet report, sending...
I20260811 22:23:41.806951 27844 ts_manager.cc:194] Registered new tserver with Master: 93021a8417ec42c4a68f7f836216b8ac (127.29.229.3:43279)
I20260811 22:23:41.807806 27844 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.229.3:44951
I20260811 22:23:41.818447 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:23:41.829873 28603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:41.823054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40258 (local address 127.29.229.62:42897)
0811 22:23:41.823309 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:41.823318 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:41.823719 (+ 401us) server_negotiation.cc:407] Connection header received
0811 22:23:41.823866 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:41.823869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:41.823935 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:41.824020 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:41.824614 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.825335 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.826951 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.827363 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.827474 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:41.829009 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:41.829014 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:41.829018 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:41.829036 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:41.829072 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:41.829084 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:41.829249 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:41.829392 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 22:23:41.829494 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:23:41.834860 27844 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:23:41.841859 27844 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:23:41.869644 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:41.856593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:40967 (local address 127.0.0.1:37382)
0811 22:23:41.858322 (+ 1729us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:41.858366 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:41.858707 (+ 341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:41.859864 (+ 1157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:41.859876 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:41.859920 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:41.860661 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:41.860685 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.863170 (+ 2485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.863176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:41.866172 (+ 2996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:41.866190 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.868665 (+ 2475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.868672 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:41.868860 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:41.868912 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 22:23:41.869048 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 22:23:41.869245 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1434,"mutex_wait_us":87,"thread_start_us":1206,"threads_started":1}
I20260811 22:23:41.869932 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:41.856771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37382 (local address 127.29.229.2:40967)
0811 22:23:41.857802 (+ 1031us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:41.857814 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:41.858522 (+ 708us) server_negotiation.cc:407] Connection header received
0811 22:23:41.859017 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:41.859026 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:41.859330 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:41.859499 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:41.860927 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.862967 (+ 2040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.866463 (+ 3496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.868046 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.868215 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:41.868531 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:41.869380 (+ 849us) server_negotiation.cc:299] Negotiation successful
0811 22:23:41.869577 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":791,"thread_start_us":592,"threads_started":1}
I20260811 22:23:41.870896 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:41.857285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:43279 (local address 127.0.0.1:36306)
0811 22:23:41.858533 (+ 1248us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:41.858555 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:41.858706 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:41.862804 (+ 4098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:41.862809 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:41.862836 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:41.863293 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:41.863308 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.865492 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.865497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:41.868049 (+ 2552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:41.868063 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.869974 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.869980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:41.870186 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:41.870273 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 22:23:41.870346 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 22:23:41.870469 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1103,"mutex_wait_us":151,"thread_start_us":1392,"threads_started":1}
I20260811 22:23:41.870913 28626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:41.858738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:41049 (local address 127.0.0.1:49722)
0811 22:23:41.861283 (+ 2545us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:41.861307 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:41.861464 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:41.863191 (+ 1727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:41.863196 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:41.863239 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:41.863853 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:41.863873 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.865704 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.865709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:41.867617 (+ 1908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:41.867633 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.869863 (+ 2230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.869872 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:41.870192 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:41.870272 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 22:23:41.870388 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 22:23:41.870549 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2373,"mutex_wait_us":79,"thread_start_us":831,"threads_started":1}
I20260811 22:23:41.871637 28629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:41.859371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49722 (local address 127.29.229.1:41049)
0811 22:23:41.862556 (+ 3185us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:41.862563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:41.862592 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:23:41.862701 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:41.862710 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:41.862960 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:41.863215 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:41.864063 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.865512 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.867814 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.869277 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.869436 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:41.869733 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:41.871036 (+ 1303us) server_negotiation.cc:299] Negotiation successful
0811 22:23:41.871245 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3018,"thread_start_us":2872,"threads_started":1}
I20260811 22:23:41.872290 28625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:41.858287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36306 (local address 127.29.229.3:43279)
0811 22:23:41.862148 (+ 3861us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:41.862159 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:41.862203 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:23:41.862317 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:41.862326 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:41.862585 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:41.862794 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:41.863591 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.865339 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.868274 (+ 2935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:41.869720 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:41.871299 (+ 1579us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:41.871588 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:41.871665 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 22:23:41.871860 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3663,"thread_start_us":3505,"threads_started":1}
I20260811 22:23:41.874976 28052 tablet_service.cc:1467] Processing CreateTablet for tablet 5aa0b299d4fa4420abc948f177db2b2c (DEFAULT_TABLE table=test-table [id=e9795079c6064638beec11175ac69fe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:41.874838 28273 tablet_service.cc:1467] Processing CreateTablet for tablet 5aa0b299d4fa4420abc948f177db2b2c (DEFAULT_TABLE table=test-table [id=e9795079c6064638beec11175ac69fe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:41.875154 28500 tablet_service.cc:1467] Processing CreateTablet for tablet 5aa0b299d4fa4420abc948f177db2b2c (DEFAULT_TABLE table=test-table [id=e9795079c6064638beec11175ac69fe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:41.876391 28500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5aa0b299d4fa4420abc948f177db2b2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:41.876410 28273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5aa0b299d4fa4420abc948f177db2b2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:41.876427 28052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5aa0b299d4fa4420abc948f177db2b2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:41.890074 28630 tablet_bootstrap.cc:492] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac: Bootstrap starting.
I20260811 22:23:41.893340 28633 tablet_bootstrap.cc:492] T 5aa0b299d4fa4420abc948f177db2b2c P 1dba52d467624ffc80a62ef0bbf66bc2: Bootstrap starting.
I20260811 22:23:41.894181 28632 tablet_bootstrap.cc:492] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914: Bootstrap starting.
I20260811 22:23:41.895516 28630 tablet_bootstrap.cc:654] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:41.896970 28630 log.cc:824] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:41.898499 28632 tablet_bootstrap.cc:654] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:41.899819 28632 log.cc:824] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:41.900441 28630 tablet_bootstrap.cc:492] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac: No bootstrap required, opened a new log
I20260811 22:23:41.900769 28630 ts_tablet_manager.cc:1397] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260811 22:23:41.901794 28633 tablet_bootstrap.cc:654] T 5aa0b299d4fa4420abc948f177db2b2c P 1dba52d467624ffc80a62ef0bbf66bc2: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:41.903470 28633 log.cc:824] T 5aa0b299d4fa4420abc948f177db2b2c P 1dba52d467624ffc80a62ef0bbf66bc2: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:41.906289 28632 tablet_bootstrap.cc:492] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914: No bootstrap required, opened a new log
I20260811 22:23:41.906574 28632 ts_tablet_manager.cc:1397] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260811 22:23:41.906652 28633 tablet_bootstrap.cc:492] T 5aa0b299d4fa4420abc948f177db2b2c P 1dba52d467624ffc80a62ef0bbf66bc2: No bootstrap required, opened a new log
I20260811 22:23:41.906921 28633 ts_tablet_manager.cc:1397] T 5aa0b299d4fa4420abc948f177db2b2c P 1dba52d467624ffc80a62ef0bbf66bc2: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.000s
I20260811 22:23:41.910269 28630 raft_consensus.cc:348] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d12a4ab1fa443cb5a915894207d914" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40967 } } peers { permanent_uuid: "1dba52d467624ffc80a62ef0bbf66bc2" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 41049 } } peers { permanent_uuid: "93021a8417ec42c4a68f7f836216b8ac" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 43279 } }
I20260811 22:23:41.910744 28630 raft_consensus.cc:374] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:41.910876 28630 raft_consensus.cc:729] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93021a8417ec42c4a68f7f836216b8ac, State: Initialized, Role: FOLLOWER
I20260811 22:23:41.911545 28630 consensus_queue.cc:260] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d12a4ab1fa443cb5a915894207d914" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40967 } } peers { permanent_uuid: "1dba52d467624ffc80a62ef0bbf66bc2" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 41049 } } peers { permanent_uuid: "93021a8417ec42c4a68f7f836216b8ac" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 43279 } }
I20260811 22:23:41.912748 28599 heartbeater.cc:503] Master 127.29.229.62:42897 was elected leader, sending a full tablet report...
I20260811 22:23:41.913424 28630 ts_tablet_manager.cc:1428] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac: Time spent starting tablet: real 0.013s user 0.005s sys 0.007s
I20260811 22:23:41.915508 28632 raft_consensus.cc:348] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d12a4ab1fa443cb5a915894207d914" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40967 } } peers { permanent_uuid: "1dba52d467624ffc80a62ef0bbf66bc2" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 41049 } } peers { permanent_uuid: "93021a8417ec42c4a68f7f836216b8ac" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 43279 } }
I20260811 22:23:41.915890 28632 raft_consensus.cc:374] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:41.916023 28632 raft_consensus.cc:729] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d12a4ab1fa443cb5a915894207d914, State: Initialized, Role: FOLLOWER
I20260811 22:23:41.916689 28632 consensus_queue.cc:260] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d12a4ab1fa443cb5a915894207d914" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40967 } } peers { permanent_uuid: "1dba52d467624ffc80a62ef0bbf66bc2" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 41049 } } peers { permanent_uuid: "93021a8417ec42c4a68f7f836216b8ac" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 43279 } }
I20260811 22:23:41.918145 28382 heartbeater.cc:503] Master 127.29.229.62:42897 was elected leader, sending a full tablet report...
I20260811 22:23:41.917970 28633 raft_consensus.cc:348] T 5aa0b299d4fa4420abc948f177db2b2c P 1dba52d467624ffc80a62ef0bbf66bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d12a4ab1fa443cb5a915894207d914" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40967 } } peers { permanent_uuid: "1dba52d467624ffc80a62ef0bbf66bc2" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 41049 } } peers { permanent_uuid: "93021a8417ec42c4a68f7f836216b8ac" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 43279 } }
I20260811 22:23:41.918394 28633 raft_consensus.cc:374] T 5aa0b299d4fa4420abc948f177db2b2c P 1dba52d467624ffc80a62ef0bbf66bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:41.918511 28633 raft_consensus.cc:729] T 5aa0b299d4fa4420abc948f177db2b2c P 1dba52d467624ffc80a62ef0bbf66bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dba52d467624ffc80a62ef0bbf66bc2, State: Initialized, Role: FOLLOWER
I20260811 22:23:41.918918 28632 ts_tablet_manager.cc:1428] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260811 22:23:41.919430 28633 consensus_queue.cc:260] T 5aa0b299d4fa4420abc948f177db2b2c P 1dba52d467624ffc80a62ef0bbf66bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d12a4ab1fa443cb5a915894207d914" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40967 } } peers { permanent_uuid: "1dba52d467624ffc80a62ef0bbf66bc2" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 41049 } } peers { permanent_uuid: "93021a8417ec42c4a68f7f836216b8ac" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 43279 } }
I20260811 22:23:41.921397 28633 ts_tablet_manager.cc:1428] T 5aa0b299d4fa4420abc948f177db2b2c P 1dba52d467624ffc80a62ef0bbf66bc2: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
W20260811 22:23:42.001820 28163 tablet.cc:2369] T 5aa0b299d4fa4420abc948f177db2b2c P 1dba52d467624ffc80a62ef0bbf66bc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:23:42.020452 28384 tablet.cc:2369] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:23:42.024310 28642 raft_consensus.cc:482] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:23:42.024698 28642 raft_consensus.cc:504] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d12a4ab1fa443cb5a915894207d914" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40967 } } peers { permanent_uuid: "1dba52d467624ffc80a62ef0bbf66bc2" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 41049 } } peers { permanent_uuid: "93021a8417ec42c4a68f7f836216b8ac" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 43279 } }
I20260811 22:23:42.026599 28642 leader_election.cc:288] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dba52d467624ffc80a62ef0bbf66bc2 (127.29.229.1:41049), 93021a8417ec42c4a68f7f836216b8ac (127.29.229.3:43279)
I20260811 22:23:42.034039 28625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:42.027142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:43661 (local address 127.29.229.3:43279)
0811 22:23:42.027393 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:42.027402 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:42.028243 (+ 841us) server_negotiation.cc:407] Connection header received
0811 22:23:42.028426 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:42.028430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:42.028549 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:42.028654 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:42.029403 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:42.030140 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:42.032203 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:42.033058 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:42.033247 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:42.033323 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:42.033662 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 22:23:42.033784 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:23:42.034119 28629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:42.027088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:38179 (local address 127.29.229.1:41049)
0811 22:23:42.027325 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:42.027334 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:42.028135 (+ 801us) server_negotiation.cc:407] Connection header received
0811 22:23:42.028410 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:42.028414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:42.028542 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:42.028656 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:42.029342 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:42.030084 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:42.032300 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:42.033171 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:42.033359 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:42.033440 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:42.033741 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 22:23:42.033867 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:23:42.034200 28645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:42.027049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:43279 (local address 127.29.229.2:43661)
0811 22:23:42.028130 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:42.028145 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:42.028263 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:42.028676 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:42.028680 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:42.028745 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:42.029224 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:42.029235 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:42.030297 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:42.030306 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:42.031995 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:42.032009 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:42.033285 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:42.033292 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:42.033456 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:42.033488 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:23:42.033601 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 22:23:42.033735 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":976,"mutex_wait_us":60,"spinlock_wait_cycles":6656,"thread_start_us":727,"threads_started":1}
I20260811 22:23:42.034248 28648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:42.026956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:41049 (local address 127.29.229.2:38179)
0811 22:23:42.028012 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:42.028028 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:42.028238 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:42.028674 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:42.028679 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:42.028741 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:42.029183 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:42.029197 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:42.030224 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:42.030231 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:42.032070 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:42.032084 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:42.033371 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:42.033377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:42.033566 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:42.033589 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:23:42.033697 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 22:23:42.033823 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":920,"mutex_wait_us":61,"spinlock_wait_cycles":53376,"thread_start_us":677,"threads_started":1}
I20260811 22:23:42.035374 28089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aa0b299d4fa4420abc948f177db2b2c" candidate_uuid: "f9d12a4ab1fa443cb5a915894207d914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dba52d467624ffc80a62ef0bbf66bc2" is_pre_election: true
I20260811 22:23:42.035374 28532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aa0b299d4fa4420abc948f177db2b2c" candidate_uuid: "f9d12a4ab1fa443cb5a915894207d914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93021a8417ec42c4a68f7f836216b8ac" is_pre_election: true
I20260811 22:23:42.036056 28089 raft_consensus.cc:2454] T 5aa0b299d4fa4420abc948f177db2b2c P 1dba52d467624ffc80a62ef0bbf66bc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9d12a4ab1fa443cb5a915894207d914 in term 0.
I20260811 22:23:42.036018 28532 raft_consensus.cc:2454] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9d12a4ab1fa443cb5a915894207d914 in term 0.
I20260811 22:23:42.036803 28197 leader_election.cc:302] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1dba52d467624ffc80a62ef0bbf66bc2, 93021a8417ec42c4a68f7f836216b8ac, f9d12a4ab1fa443cb5a915894207d914; no voters:
I20260811 22:23:42.037253 28642 raft_consensus.cc:2781] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:23:42.037416 28642 raft_consensus.cc:482] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:23:42.037524 28642 raft_consensus.cc:3037] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [term 0 FOLLOWER]: Advancing to term 1
W20260811 22:23:42.038674 28602 tablet.cc:2369] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:23:42.039750 28642 raft_consensus.cc:504] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d12a4ab1fa443cb5a915894207d914" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40967 } } peers { permanent_uuid: "1dba52d467624ffc80a62ef0bbf66bc2" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 41049 } } peers { permanent_uuid: "93021a8417ec42c4a68f7f836216b8ac" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 43279 } }
I20260811 22:23:42.040397 28642 leader_election.cc:288] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [CANDIDATE]: Term 1 election: Requested vote from peers 1dba52d467624ffc80a62ef0bbf66bc2 (127.29.229.1:41049), 93021a8417ec42c4a68f7f836216b8ac (127.29.229.3:43279)
I20260811 22:23:42.040944 28089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aa0b299d4fa4420abc948f177db2b2c" candidate_uuid: "f9d12a4ab1fa443cb5a915894207d914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dba52d467624ffc80a62ef0bbf66bc2"
I20260811 22:23:42.040951 28532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aa0b299d4fa4420abc948f177db2b2c" candidate_uuid: "f9d12a4ab1fa443cb5a915894207d914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93021a8417ec42c4a68f7f836216b8ac"
I20260811 22:23:42.041167 28089 raft_consensus.cc:3037] T 5aa0b299d4fa4420abc948f177db2b2c P 1dba52d467624ffc80a62ef0bbf66bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:42.041436 28532 raft_consensus.cc:3037] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:42.043542 28089 raft_consensus.cc:2454] T 5aa0b299d4fa4420abc948f177db2b2c P 1dba52d467624ffc80a62ef0bbf66bc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d12a4ab1fa443cb5a915894207d914 in term 1.
I20260811 22:23:42.044057 28197 leader_election.cc:302] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dba52d467624ffc80a62ef0bbf66bc2, f9d12a4ab1fa443cb5a915894207d914; no voters:
I20260811 22:23:42.044378 28642 raft_consensus.cc:2781] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:23:42.044950 28532 raft_consensus.cc:2454] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d12a4ab1fa443cb5a915894207d914 in term 1.
I20260811 22:23:42.045076 28642 raft_consensus.cc:686] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [term 1 LEADER]: Becoming Leader. State: Replica: f9d12a4ab1fa443cb5a915894207d914, State: Running, Role: LEADER
I20260811 22:23:42.045656 28642 consensus_queue.cc:237] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d12a4ab1fa443cb5a915894207d914" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40967 } } peers { permanent_uuid: "1dba52d467624ffc80a62ef0bbf66bc2" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 41049 } } peers { permanent_uuid: "93021a8417ec42c4a68f7f836216b8ac" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 43279 } }
I20260811 22:23:42.050598 27844 catalog_manager.cc:5495] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9d12a4ab1fa443cb5a915894207d914 (127.29.229.2). New cstate: current_term: 1 leader_uuid: "f9d12a4ab1fa443cb5a915894207d914" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d12a4ab1fa443cb5a915894207d914" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dba52d467624ffc80a62ef0bbf66bc2" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93021a8417ec42c4a68f7f836216b8ac" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 43279 } health_report { overall_health: UNKNOWN } } }
I20260811 22:23:42.113898 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:42.108073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37390 (local address 127.29.229.2:40967)
0811 22:23:42.108295 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:42.108300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:42.108319 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:23:42.108423 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:42.108430 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:42.108576 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:42.108708 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:23:42.109062 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:42.109783 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:42.111388 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:42.111849 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:42.111957 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:42.112446 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:23:42.113124 (+ 678us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:23:42.113241 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:42.113443 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 22:23:42.113531 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 22:23:42.124274 28089 raft_consensus.cc:1261] T 5aa0b299d4fa4420abc948f177db2b2c P 1dba52d467624ffc80a62ef0bbf66bc2 [term 1 FOLLOWER]: Refusing update from remote peer f9d12a4ab1fa443cb5a915894207d914: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:23:42.124396 28532 raft_consensus.cc:1261] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac [term 1 FOLLOWER]: Refusing update from remote peer f9d12a4ab1fa443cb5a915894207d914: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:23:42.125422 28642 consensus_queue.cc:1035] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dba52d467624ffc80a62ef0bbf66bc2" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:23:42.125783 28649 consensus_queue.cc:1035] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93021a8417ec42c4a68f7f836216b8ac" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 43279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:23:42.155743 27843 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:23:42.156062 27843 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:23:42.160043 27843 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f2bca6d435644c46b3bc018c65ec2ed7: Sending DeleteTablet for 3 replicas of tablet 5aa0b299d4fa4420abc948f177db2b2c
I20260811 22:23:42.161363 28273 tablet_service.cc:1514] Processing DeleteTablet for tablet 5aa0b299d4fa4420abc948f177db2b2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:42 UTC) from {username='slave'} at 127.0.0.1:37382
I20260811 22:23:42.161458 28500 tablet_service.cc:1514] Processing DeleteTablet for tablet 5aa0b299d4fa4420abc948f177db2b2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:42 UTC) from {username='slave'} at 127.0.0.1:36306
I20260811 22:23:42.161486 28052 tablet_service.cc:1514] Processing DeleteTablet for tablet 5aa0b299d4fa4420abc948f177db2b2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:42 UTC) from {username='slave'} at 127.0.0.1:49722
I20260811 22:23:42.165464 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 27945
I20260811 22:23:42.173933 28677 tablet_replica.cc:330] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac: stopping tablet replica
I20260811 22:23:42.174542 28677 raft_consensus.cc:2229] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:23:42.175125 28677 raft_consensus.cc:2258] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:42.173043 28675 tablet_replica.cc:330] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914: stopping tablet replica
I20260811 22:23:42.176038 28675 raft_consensus.cc:2229] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:23:42.176788 28675 raft_consensus.cc:2258] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:42.177640 28676 tablet_replica.cc:330] T 5aa0b299d4fa4420abc948f177db2b2c P 1dba52d467624ffc80a62ef0bbf66bc2: stopping tablet replica
I20260811 22:23:42.178050 28676 raft_consensus.cc:2229] T 5aa0b299d4fa4420abc948f177db2b2c P 1dba52d467624ffc80a62ef0bbf66bc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:23:42.178578 28676 raft_consensus.cc:2258] T 5aa0b299d4fa4420abc948f177db2b2c P 1dba52d467624ffc80a62ef0bbf66bc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:42.179682 28677 ts_tablet_manager.cc:1905] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:42.181025 28675 ts_tablet_manager.cc:1905] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:42.182574 28676 ts_tablet_manager.cc:1905] T 5aa0b299d4fa4420abc948f177db2b2c P 1dba52d467624ffc80a62ef0bbf66bc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:42.184616 28629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:42.167957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49738 (local address 127.29.229.1:41049)
0811 22:23:42.168212 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:42.168221 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:42.173103 (+ 4882us) server_negotiation.cc:407] Connection header received
0811 22:23:42.173544 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:42.173554 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:42.173725 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:42.173825 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:42.174853 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:42.176444 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:42.178834 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:42.179484 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:42.179585 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:42.182839 (+ 3254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:42.182851 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:42.182868 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:42.183623 (+ 755us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:42.183683 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:42.183708 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:42.183799 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:42.184054 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 22:23:42.184183 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:23:42.185765 28156 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:42.187772 28677 ts_tablet_manager.cc:1918] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:23:42.187988 28677 log.cc:1196] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/5aa0b299d4fa4420abc948f177db2b2c
I20260811 22:23:42.188740 28677 ts_tablet_manager.cc:1939] T 5aa0b299d4fa4420abc948f177db2b2c P 93021a8417ec42c4a68f7f836216b8ac: Deleting consensus metadata
I20260811 22:23:42.190814 28675 ts_tablet_manager.cc:1918] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:23:42.191031 28675 log.cc:1196] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/5aa0b299d4fa4420abc948f177db2b2c
I20260811 22:23:42.191710 28675 ts_tablet_manager.cc:1939] T 5aa0b299d4fa4420abc948f177db2b2c P f9d12a4ab1fa443cb5a915894207d914: Deleting consensus metadata
I20260811 22:23:42.195204 27827 catalog_manager.cc:4841] TS 93021a8417ec42c4a68f7f836216b8ac (127.29.229.3:43279): tablet 5aa0b299d4fa4420abc948f177db2b2c (table test-table [id=e9795079c6064638beec11175ac69fe9]) successfully deleted
I20260811 22:23:42.196390 27830 catalog_manager.cc:4841] TS f9d12a4ab1fa443cb5a915894207d914 (127.29.229.2:40967): tablet 5aa0b299d4fa4420abc948f177db2b2c (table test-table [id=e9795079c6064638beec11175ac69fe9]) successfully deleted
I20260811 22:23:42.296422 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 27945
W20260811 22:23:42.330453 27830 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.229.1:41049 (error 108)
I20260811 22:23:42.331313 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 28172
I20260811 22:23:42.332630 28626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:42.331991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:41049 (local address 127.0.0.1:49754)
0811 22:23:42.332316 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:42.332424 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:41049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208}
W20260811 22:23:42.332907 27830 catalog_manager.cc:4584] TS 1dba52d467624ffc80a62ef0bbf66bc2 (127.29.229.1:41049): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5aa0b299d4fa4420abc948f177db2b2c: Network error: Client connection negotiation failed: client connection to 127.29.229.1:41049: connect: Connection refused (error 111)
I20260811 22:23:42.340091 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:42.332079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37396 (local address 127.29.229.2:40967)
0811 22:23:42.332315 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:42.332324 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:42.332949 (+ 625us) server_negotiation.cc:407] Connection header received
0811 22:23:42.333024 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:42.333030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:42.333140 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:42.333251 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:42.333662 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:42.334892 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:42.336253 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:42.336996 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:42.337140 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:42.338343 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:42.338354 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:42.338374 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:42.339207 (+ 833us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:42.339276 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:42.339304 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:42.339423 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:42.339588 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 22:23:42.339705 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 22:23:42.340874 28375 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:42.386850 28626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:42.386281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:41049 (local address 127.0.0.1:49764)
0811 22:23:42.386523 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:42.386635 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:41049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 22:23:42.387965 28626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:42.387575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:41049 (local address 127.0.0.1:49780)
0811 22:23:42.387738 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:42.387819 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:41049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 22:23:42.448812 28626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:42.448198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:41049 (local address 127.0.0.1:49796)
0811 22:23:42.448454 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:42.448589 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:41049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 22:23:42.449882 28626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:42.449481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:41049 (local address 127.0.0.1:49800)
0811 22:23:42.449649 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:42.449731 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:41049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 22:23:42.460996 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 28172
I20260811 22:23:42.491333 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 28392
I20260811 22:23:42.499022 28625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:42.492197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36320 (local address 127.29.229.3:43279)
0811 22:23:42.492419 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:42.492426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:42.492575 (+ 149us) server_negotiation.cc:407] Connection header received
0811 22:23:42.492781 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:42.492791 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:42.492885 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:42.492995 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:42.493567 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:42.494298 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:42.495571 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:42.495987 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:42.496125 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:42.497622 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:42.497631 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:42.497646 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:42.498229 (+ 583us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:42.498277 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:42.498302 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:42.498432 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:42.498576 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 22:23:42.498692 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 22:23:42.499722 28593 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:42.530458 28626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:42.529945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:41049 (local address 127.0.0.1:49814)
0811 22:23:42.530167 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:42.530264 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:41049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 22:23:42.532044 28626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:42.531636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:41049 (local address 127.0.0.1:49826)
0811 22:23:42.531856 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:42.531939 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:41049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260811 22:23:42.594745 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 28392
I20260811 22:23:42.621413 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 27795
I20260811 22:23:42.622572 27900 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:42.736079 28626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:42.735115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:41049 (local address 127.0.0.1:49838)
0811 22:23:42.735558 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:42.735696 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:41049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265}
I20260811 22:23:42.736572 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 27795
2026-08-11T22:23:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3216 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:23:42 dist-test-slave-jq4t krb5kdc[28693](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:23:42 dist-test-slave-jq4t krb5kdc[28693](info): set up 2 sockets
Aug 11 22:23:42 dist-test-slave-jq4t krb5kdc[28693](info): commencing operation
krb5kdc: starting...
W20260811 22:23:44.158435 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.368s user 0.010s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:23:44 dist-test-slave-jq4t krb5kdc[28693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:23:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:23:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:23:44.549050 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:37829
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:44289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:37829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:23:44.927781 28712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:44.928098 28712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:44.928261 28712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:44.935787 28712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:23:44.935923 28712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:44.935971 28712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:44.936087 28712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:23:44.936157 28712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:23:44.946493 28712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:44289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:37829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:37829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:44.947916 28712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:44.949316 28712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:44.961623 28724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:44.961649 28719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:44.961589 28721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:44.962744 28712 server_base.cc:1034] running on GCE node
I20260811 22:23:44.963593 28712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:44.965070 28712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:44.966301 28712 hybrid_clock.cc:648] HybridClock initialized: now 1786487024966243 us; error 42 us; skew 500 ppm
I20260811 22:23:44.966639 28712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:23:44 dist-test-slave-jq4t krb5kdc[28693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487024, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:44.975382 28712 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:23:44.977358 28712 webserver.cc:460] Webserver started at http://127.29.229.62:45195/ using document root <none> and password file <none>
I20260811 22:23:44.978098 28712 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:44.978206 28712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:44.978477 28712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:44.981832 28712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "e35a3f3ebff0421dab00c2a597eef1ae"
format_stamp: "Formatted at 2026-08-11 22:23:44 on dist-test-slave-jq4t"
server_key: "1511ea45aacda9582b7a90ccc17a56ec"
server_key_iv: "1ceb9f8b64a6009150dc146d654a7afe"
server_key_version: "encryptionkey@0"
I20260811 22:23:44.982633 28712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "e35a3f3ebff0421dab00c2a597eef1ae"
format_stamp: "Formatted at 2026-08-11 22:23:44 on dist-test-slave-jq4t"
server_key: "1511ea45aacda9582b7a90ccc17a56ec"
server_key_iv: "1ceb9f8b64a6009150dc146d654a7afe"
server_key_version: "encryptionkey@0"
I20260811 22:23:44.989733 28712 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260811 22:23:44.994516 28732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:44.996788 28712 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:23:44.997171 28712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "e35a3f3ebff0421dab00c2a597eef1ae"
format_stamp: "Formatted at 2026-08-11 22:23:44 on dist-test-slave-jq4t"
server_key: "1511ea45aacda9582b7a90ccc17a56ec"
server_key_iv: "1ceb9f8b64a6009150dc146d654a7afe"
server_key_version: "encryptionkey@0"
I20260811 22:23:44.997615 28712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:45.017230 28712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:45.022903 28712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:45.023327 28712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:45.045783 28712 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:37829
I20260811 22:23:45.045822 28821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:37829 every 8 connection(s)
I20260811 22:23:45.047324 28712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:23:45.049629 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 28712
I20260811 22:23:45.049932 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:23:45.050454 30612 external_mini_cluster.cc:1428] Setting key 3f3bc06f80e783720150bae6eb507cc6
I20260811 22:23:45.051613 28824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 22:23:45 dist-test-slave-jq4t krb5kdc[28693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:23:45.067677 28824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae: Bootstrap starting.
I20260811 22:23:45.071522 28824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:45.072760 28824 log.cc:824] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:45.075677 28824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae: No bootstrap required, opened a new log
I20260811 22:23:45.077441 28833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:45.052822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.29.229.62:37829)
0811 22:23:45.053686 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:45.053698 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:45.053740 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:23:45.055362 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:45.055384 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:45.055792 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:45.056134 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:45.058211 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:45.059651 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:45.061633 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:45.062344 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:45.062500 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:45.068126 (+ 5626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:45.068136 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:45.068151 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:45.068194 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:45.072677 (+ 4483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:45.073649 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:45.073657 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:45.073667 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:45.073886 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:45.074329 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:45.074333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:45.074335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:45.074891 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:45.075420 (+ 529us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:45.075908 (+ 488us) server_negotiation.cc:299] Negotiation successful
0811 22:23:45.076391 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":406,"threads_started":1}
I20260811 22:23:45.084205 28824 raft_consensus.cc:348] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35a3f3ebff0421dab00c2a597eef1ae" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37829 } }
I20260811 22:23:45.084681 28824 raft_consensus.cc:374] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:45.084791 28824 raft_consensus.cc:729] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e35a3f3ebff0421dab00c2a597eef1ae, State: Initialized, Role: FOLLOWER
I20260811 22:23:45.085430 28824 consensus_queue.cc:260] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35a3f3ebff0421dab00c2a597eef1ae" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37829 } }
I20260811 22:23:45.085682 28824 raft_consensus.cc:388] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:23:45.085836 28824 raft_consensus.cc:482] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:23:45.086040 28824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:45.088361 28824 raft_consensus.cc:504] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35a3f3ebff0421dab00c2a597eef1ae" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37829 } }
I20260811 22:23:45.088927 28824 leader_election.cc:302] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e35a3f3ebff0421dab00c2a597eef1ae; no voters:
I20260811 22:23:45.089686 28824 leader_election.cc:288] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:23:45.090067 28837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:23:45.091768 28837 raft_consensus.cc:686] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae [term 1 LEADER]: Becoming Leader. State: Replica: e35a3f3ebff0421dab00c2a597eef1ae, State: Running, Role: LEADER
I20260811 22:23:45.092682 28837 consensus_queue.cc:237] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35a3f3ebff0421dab00c2a597eef1ae" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37829 } }
I20260811 22:23:45.092985 28824 sys_catalog.cc:564] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:23:45.095461 28839 sys_catalog.cc:455] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader e35a3f3ebff0421dab00c2a597eef1ae. Latest consensus state: current_term: 1 leader_uuid: "e35a3f3ebff0421dab00c2a597eef1ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35a3f3ebff0421dab00c2a597eef1ae" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37829 } } }
I20260811 22:23:45.095810 28839 sys_catalog.cc:458] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:45.096427 28838 sys_catalog.cc:455] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e35a3f3ebff0421dab00c2a597eef1ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35a3f3ebff0421dab00c2a597eef1ae" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 37829 } } }
I20260811 22:23:45.096769 28838 sys_catalog.cc:458] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:45.097177 28853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:23:45.104103 28853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:23:45.113770 28853 catalog_manager.cc:1347] Generated new cluster ID: e2b76633d51145c7ad960e181b1e470b
I20260811 22:23:45.113924 28853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:23:45.134012 28853 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:23:45.135612 28853 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:23:45.150122 28853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae: Generated new TSK 0
I20260811 22:23:45.151047 28853 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
I20260811 22:23:45.242746 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:37829
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:44289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:23:45.609767 28871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:45.610067 28871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:45.610234 28871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:45.617700 28871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:45.617832 28871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:45.618038 28871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:23:45.628427 28871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:44289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:37829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:45.629874 28871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:45.631299 28871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:45.646803 28883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:45.647008 28878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:45.647014 28880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:45.647948 28871 server_base.cc:1034] running on GCE node
I20260811 22:23:45.648720 28871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:45.649930 28871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:45.651346 28871 hybrid_clock.cc:648] HybridClock initialized: now 1786487025651299 us; error 66 us; skew 500 ppm
I20260811 22:23:45.651757 28871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:23:45 dist-test-slave-jq4t krb5kdc[28693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487025, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:45.655402 28871 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:23:45.657294 28871 webserver.cc:460] Webserver started at http://127.29.229.1:44263/ using document root <none> and password file <none>
I20260811 22:23:45.658111 28871 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:45.658277 28871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:45.658635 28871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:45.661908 28871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "c21bb9ea81e74a89bd385601678c5511"
format_stamp: "Formatted at 2026-08-11 22:23:45 on dist-test-slave-jq4t"
server_key: "31fbaf84d94a94ba5fd3f7f4747c757b"
server_key_iv: "e73ea7c365304686b4942a0cb7ab94c7"
server_key_version: "encryptionkey@0"
I20260811 22:23:45.662755 28871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "c21bb9ea81e74a89bd385601678c5511"
format_stamp: "Formatted at 2026-08-11 22:23:45 on dist-test-slave-jq4t"
server_key: "31fbaf84d94a94ba5fd3f7f4747c757b"
server_key_iv: "e73ea7c365304686b4942a0cb7ab94c7"
server_key_version: "encryptionkey@0"
I20260811 22:23:45.670395 28871 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260811 22:23:45.675060 28893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:45.677423 28871 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:23:45.677747 28871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "c21bb9ea81e74a89bd385601678c5511"
format_stamp: "Formatted at 2026-08-11 22:23:45 on dist-test-slave-jq4t"
server_key: "31fbaf84d94a94ba5fd3f7f4747c757b"
server_key_iv: "e73ea7c365304686b4942a0cb7ab94c7"
server_key_version: "encryptionkey@0"
I20260811 22:23:45.678074 28871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:45.721832 28871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:45.726524 28871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:45.726878 28871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:45.727970 28871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:45.729473 28871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:45.729593 28871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:45.729749 28871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:45.729825 28871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:45.770567 28871 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:36029
I20260811 22:23:45.770609 29076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:36029 every 8 connection(s)
I20260811 22:23:45.771997 28871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:23:45.772672 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 28871
I20260811 22:23:45.772898 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:23:45.773489 30612 external_mini_cluster.cc:1428] Setting key 1bd185aef360be9075f9ddde5e565f51
Aug 11 22:23:45 dist-test-slave-jq4t krb5kdc[28693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487025, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:23:45.802412 29081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:45.775010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:45651 (local address 127.29.229.62:37829)
0811 22:23:45.776187 (+ 1177us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:45.776196 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:45.777327 (+ 1131us) server_negotiation.cc:407] Connection header received
0811 22:23:45.780327 (+ 3000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:45.780331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:45.780410 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:45.780616 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:45.783031 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:45.784802 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:45.787727 (+ 2925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:45.788213 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:45.788337 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:45.794030 (+ 5693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:45.794037 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:45.794040 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:45.794096 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:45.797927 (+ 3831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:45.799215 (+ 1288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:45.799222 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:45.799225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:45.799417 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:45.800089 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:45.800094 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:45.800097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:45.800445 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:45.800672 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:45.801793 (+ 1121us) server_negotiation.cc:299] Negotiation successful
0811 22:23:45.802018 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1009,"thread_start_us":856,"threads_started":1}
I20260811 22:23:45.804657 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:45.775519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:37829 (local address 127.29.229.1:45651)
0811 22:23:45.777130 (+ 1611us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:45.777197 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:45.779986 (+ 2789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:45.780871 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:45.780886 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:45.781735 (+ 849us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:45.782602 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:45.782631 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:45.785149 (+ 2518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:45.785164 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:45.787485 (+ 2321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:45.787509 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:45.788532 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:45.788543 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:45.788898 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:45.789792 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:45.789829 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:45.793698 (+ 3869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:45.798171 (+ 4473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:45.798186 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:45.798210 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:45.798994 (+ 784us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:45.799599 (+ 605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:45.799606 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:45.799613 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:45.799901 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:45.800700 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:45.800712 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:45.801223 (+ 511us) client_negotiation.cc:769] Sending connection context
0811 22:23:45.801655 (+ 432us) client_negotiation.cc:240] Negotiation successful
0811 22:23:45.802879 (+ 1224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1305,"thread_start_us":1034,"threads_started":1}
I20260811 22:23:45.806967 29077 heartbeater.cc:347] Connected to a master server at 127.29.229.62:37829
I20260811 22:23:45.807538 29077 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:45.808736 29077 heartbeater.cc:511] Master 127.29.229.62:37829 requested a full tablet report, sending...
I20260811 22:23:45.811573 28761 ts_manager.cc:194] Registered new tserver with Master: c21bb9ea81e74a89bd385601678c5511 (127.29.229.1:36029)
I20260811 22:23:45.813215 28761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:45651
WARNING: no policy specified for kudu/127.29.229.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
I20260811 22:23:45.868366 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:37829
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:44289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:23:46.262279 29086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:46.262645 29086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:46.262825 29086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:46.272792 29086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:46.273031 29086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:46.273228 29086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:23:46.285183 29086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:44289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:37829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:46.286664 29086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:46.288067 29086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:46.302899 29091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:46.302994 29092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:46.303244 29099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:46.310279 29086 server_base.cc:1034] running on GCE node
I20260811 22:23:46.311177 29086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:46.312315 29086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:46.313618 29086 hybrid_clock.cc:648] HybridClock initialized: now 1786487026313559 us; error 88 us; skew 500 ppm
I20260811 22:23:46.313983 29086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:23:46 dist-test-slave-jq4t krb5kdc[28693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487026, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:46.317569 29086 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:23:46.319312 29086 webserver.cc:460] Webserver started at http://127.29.229.2:39731/ using document root <none> and password file <none>
I20260811 22:23:46.320053 29086 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:46.320159 29086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:46.320429 29086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:46.323501 29086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "a46afe5a8b06494aab40f25cfdcae302"
format_stamp: "Formatted at 2026-08-11 22:23:46 on dist-test-slave-jq4t"
server_key: "e3c197e35055301d2a76d6a6a7b25784"
server_key_iv: "81399289a2b5334c3c035459a55b55ff"
server_key_version: "encryptionkey@0"
I20260811 22:23:46.324285 29086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "a46afe5a8b06494aab40f25cfdcae302"
format_stamp: "Formatted at 2026-08-11 22:23:46 on dist-test-slave-jq4t"
server_key: "e3c197e35055301d2a76d6a6a7b25784"
server_key_iv: "81399289a2b5334c3c035459a55b55ff"
server_key_version: "encryptionkey@0"
I20260811 22:23:46.331158 29086 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 22:23:46.335861 29110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:46.337697 29086 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:23:46.337879 29086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "a46afe5a8b06494aab40f25cfdcae302"
format_stamp: "Formatted at 2026-08-11 22:23:46 on dist-test-slave-jq4t"
server_key: "e3c197e35055301d2a76d6a6a7b25784"
server_key_iv: "81399289a2b5334c3c035459a55b55ff"
server_key_version: "encryptionkey@0"
I20260811 22:23:46.338073 29086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:46.355811 29086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:46.360664 29086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:46.360971 29086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:46.362207 29086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:46.363904 29086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:46.363987 29086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:46.364145 29086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:46.364204 29086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:46.408650 29086 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:40941
I20260811 22:23:46.408672 29285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:40941 every 8 connection(s)
I20260811 22:23:46.410115 29086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:23:46.412568 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 29086
I20260811 22:23:46.412786 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:23:46.413641 30612 external_mini_cluster.cc:1428] Setting key c9ebbdc97a7f1a37005cfc8c8d987dae
Aug 11 22:23:46 dist-test-slave-jq4t krb5kdc[28693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487026, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:23:46.437661 29289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:46.413173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:38121 (local address 127.29.229.62:37829)
0811 22:23:46.413672 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:46.413677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:46.415253 (+ 1576us) server_negotiation.cc:407] Connection header received
0811 22:23:46.417503 (+ 2250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:46.417517 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:46.417623 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:46.417751 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:46.419468 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:46.420658 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:46.422303 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:46.423029 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:46.423172 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:46.428355 (+ 5183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:46.428368 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:46.428376 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:46.428451 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:46.433058 (+ 4607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:46.434072 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:46.434080 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:46.434086 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:46.434254 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:46.435069 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:46.435076 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:46.435081 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:46.435418 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:46.435553 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:46.437136 (+ 1583us) server_negotiation.cc:299] Negotiation successful
0811 22:23:46.437368 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":281,"threads_started":1}
I20260811 22:23:46.439250 29290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:46.413974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:37829 (local address 127.29.229.2:38121)
0811 22:23:46.415047 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:46.415115 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:46.417204 (+ 2089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:46.418024 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:46.418033 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:46.418616 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:46.419277 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:46.419298 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:46.420841 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:46.420845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:46.422144 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:46.422158 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:46.423298 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:46.423304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:46.423550 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:46.424622 (+ 1072us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:46.424659 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:46.427956 (+ 3297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:46.433278 (+ 5322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:46.433288 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:46.433304 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:46.433921 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:46.434496 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:46.434503 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:46.434506 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:46.434886 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:46.435607 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:46.435622 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:46.436543 (+ 921us) client_negotiation.cc:769] Sending connection context
0811 22:23:46.436976 (+ 433us) client_negotiation.cc:240] Negotiation successful
0811 22:23:46.437675 (+ 699us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":395,"threads_started":1}
I20260811 22:23:46.440992 29286 heartbeater.cc:347] Connected to a master server at 127.29.229.62:37829
I20260811 22:23:46.441327 29286 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:46.442643 29286 heartbeater.cc:511] Master 127.29.229.62:37829 requested a full tablet report, sending...
I20260811 22:23:46.445284 28761 ts_manager.cc:194] Registered new tserver with Master: a46afe5a8b06494aab40f25cfdcae302 (127.29.229.2:40941)
I20260811 22:23:46.446740 28761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:38121
WARNING: no policy specified for kudu/127.29.229.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
I20260811 22:23:46.502805 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:37829
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:44289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:23:46.817625 29077 heartbeater.cc:503] Master 127.29.229.62:37829 was elected leader, sending a full tablet report...
W20260811 22:23:46.883641 29294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:46.884166 29294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:46.884363 29294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:46.893364 29294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:46.893559 29294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:46.893784 29294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:23:46.905274 29294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:44289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:37829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:46.906806 29294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:46.908203 29294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:46.922624 29300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:46.922644 29299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:46.922803 29302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:46.923507 29294 server_base.cc:1034] running on GCE node
I20260811 22:23:46.924428 29294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:46.925540 29294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:46.926796 29294 hybrid_clock.cc:648] HybridClock initialized: now 1786487026926738 us; error 44 us; skew 500 ppm
I20260811 22:23:46.927150 29294 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:23:46 dist-test-slave-jq4t krb5kdc[28693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487026, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:46.934808 29294 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:23:46.936682 29294 webserver.cc:460] Webserver started at http://127.29.229.3:32911/ using document root <none> and password file <none>
I20260811 22:23:46.937413 29294 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:46.937577 29294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:46.937922 29294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:46.941231 29294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "42541e4a02334d9eacc5d73633059d75"
format_stamp: "Formatted at 2026-08-11 22:23:46 on dist-test-slave-jq4t"
server_key: "a83d1ee8f0e359a9424b66a4a3d4268d"
server_key_iv: "0e2d8f3623e7ae53c0cf49e026a07782"
server_key_version: "encryptionkey@0"
I20260811 22:23:46.942059 29294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "42541e4a02334d9eacc5d73633059d75"
format_stamp: "Formatted at 2026-08-11 22:23:46 on dist-test-slave-jq4t"
server_key: "a83d1ee8f0e359a9424b66a4a3d4268d"
server_key_iv: "0e2d8f3623e7ae53c0cf49e026a07782"
server_key_version: "encryptionkey@0"
I20260811 22:23:46.947698 29294 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 22:23:46.951431 29312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:46.953547 29294 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:23:46.953747 29294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "42541e4a02334d9eacc5d73633059d75"
format_stamp: "Formatted at 2026-08-11 22:23:46 on dist-test-slave-jq4t"
server_key: "a83d1ee8f0e359a9424b66a4a3d4268d"
server_key_iv: "0e2d8f3623e7ae53c0cf49e026a07782"
server_key_version: "encryptionkey@0"
I20260811 22:23:46.953986 29294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:46.968346 29294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:46.973114 29294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:46.973474 29294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:46.974545 29294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:46.975934 29294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:46.976043 29294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:46.976202 29294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:46.976282 29294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:47.013816 29294 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:35371
I20260811 22:23:47.013933 29495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:35371 every 8 connection(s)
I20260811 22:23:47.015231 29294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:23:47.018307 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 29294
I20260811 22:23:47.018548 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:23:47.019479 30612 external_mini_cluster.cc:1428] Setting key 821734c2dac9738368614c8e89fe0ca7
Aug 11 22:23:47 dist-test-slave-jq4t krb5kdc[28693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487026, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:23:47.039429 29500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:47.018265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:49871 (local address 127.29.229.62:37829)
0811 22:23:47.018963 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:47.018970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:47.020027 (+ 1057us) server_negotiation.cc:407] Connection header received
0811 22:23:47.022383 (+ 2356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:47.022387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:47.022486 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:47.022622 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:47.024413 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.025267 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.026886 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.027291 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.027430 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:47.031214 (+ 3784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:47.031219 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:47.031222 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:47.031258 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:47.034643 (+ 3385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:47.036097 (+ 1454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:47.036102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:47.036106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:47.036264 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:47.037058 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:47.037065 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:47.037069 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:47.037559 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:47.037774 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:47.038905 (+ 1131us) server_negotiation.cc:299] Negotiation successful
0811 22:23:47.039125 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":351,"threads_started":1}
I20260811 22:23:47.040395 29502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:47.018857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:37829 (local address 127.29.229.3:49871)
0811 22:23:47.019790 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:47.019851 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:47.022048 (+ 2197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:47.022886 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:47.022896 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:47.023617 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:47.024229 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:47.024247 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.025431 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.025435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:47.026723 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:47.026736 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.027543 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.027551 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:47.027785 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:47.028591 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:47.028615 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:47.030939 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:47.034930 (+ 3991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:47.034948 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:47.034980 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:47.035871 (+ 891us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:47.036416 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:47.036422 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:47.036426 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:47.036796 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:47.037794 (+ 998us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:47.037804 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:47.038423 (+ 619us) client_negotiation.cc:769] Sending connection context
0811 22:23:47.038719 (+ 296us) client_negotiation.cc:240] Negotiation successful
0811 22:23:47.039185 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":387,"threads_started":1}
I20260811 22:23:47.042299 29496 heartbeater.cc:347] Connected to a master server at 127.29.229.62:37829
I20260811 22:23:47.042860 29496 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:47.043776 29496 heartbeater.cc:511] Master 127.29.229.62:37829 requested a full tablet report, sending...
I20260811 22:23:47.045569 28761 ts_manager.cc:194] Registered new tserver with Master: 42541e4a02334d9eacc5d73633059d75 (127.29.229.3:35371)
I20260811 22:23:47.046455 28761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:49871
I20260811 22:23:47.049786 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:23:47.068444 29500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:47.054372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42928 (local address 127.29.229.62:37829)
0811 22:23:47.054566 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:47.054575 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:47.054977 (+ 402us) server_negotiation.cc:407] Connection header received
0811 22:23:47.055184 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:47.055189 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:47.055278 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:47.055390 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:47.056992 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.057794 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.059169 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.059588 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.059702 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:47.062781 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:47.062786 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:47.062790 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:47.062833 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:47.065509 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:47.066313 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:47.066317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:47.066320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:47.066445 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:47.066956 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:47.066960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:47.066962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:47.067324 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:47.067550 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:47.067897 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 22:23:47.068055 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:23:47.073148 28761 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:23:47.080296 28761 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:23:47.110822 29520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:47.094190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:36029 (local address 127.0.0.1:44366)
0811 22:23:47.097255 (+ 3065us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:47.097270 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:47.097407 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:47.098178 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:47.098188 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:47.098232 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:47.098770 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:47.098786 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.101298 (+ 2512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.101303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:47.105843 (+ 4540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:47.105865 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.109510 (+ 3645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.109518 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:47.109775 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:47.109839 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 22:23:47.110031 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 22:23:47.110300 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2952,"mutex_wait_us":3,"thread_start_us":1665,"threads_started":1}
I20260811 22:23:47.111021 29522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:47.093961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:35371 (local address 127.0.0.1:55300)
0811 22:23:47.096334 (+ 2373us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:47.096397 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:47.096650 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:47.101632 (+ 4982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:47.101637 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:47.101669 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:47.102095 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:47.102105 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.104705 (+ 2600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.104713 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:47.107844 (+ 3131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:47.107865 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.109981 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.109987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:47.110233 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:47.110296 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 22:23:47.110385 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 22:23:47.110546 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2052,"mutex_wait_us":198,"thread_start_us":3051,"threads_started":1}
I20260811 22:23:47.112089 29526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:47.095003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44366 (local address 127.29.229.1:36029)
0811 22:23:47.095875 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:47.095885 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:47.097378 (+ 1493us) server_negotiation.cc:407] Connection header received
0811 22:23:47.097609 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:47.097621 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:47.097869 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:47.098142 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:47.098990 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.101153 (+ 2163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.106093 (+ 4940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.108864 (+ 2771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.109060 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:47.109367 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:47.111446 (+ 2079us) server_negotiation.cc:299] Negotiation successful
0811 22:23:47.111671 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":431,"threads_started":1}
I20260811 22:23:47.113725 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:47.096613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:40941 (local address 127.0.0.1:38824)
0811 22:23:47.099674 (+ 3061us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:47.099692 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:47.099810 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:47.100745 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:47.100750 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:47.100778 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:47.101146 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:47.101155 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.104859 (+ 3704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.104866 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:47.110215 (+ 5349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:47.110230 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.112867 (+ 2637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.112873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:47.113046 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:47.113089 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 22:23:47.113182 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 22:23:47.113329 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2949,"mutex_wait_us":2410,"thread_start_us":2784,"threads_started":1}
I20260811 22:23:47.115012 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:47.096973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38824 (local address 127.29.229.2:40941)
0811 22:23:47.097897 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:47.097908 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:47.099812 (+ 1904us) server_negotiation.cc:407] Connection header received
0811 22:23:47.100188 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:47.100201 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:47.100545 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:47.100744 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:47.101871 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.104663 (+ 2792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.110439 (+ 5776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.112314 (+ 1875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.112448 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:47.112669 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:47.114178 (+ 1509us) server_negotiation.cc:299] Negotiation successful
0811 22:23:47.114530 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":769,"thread_start_us":833,"threads_started":1}
I20260811 22:23:47.116029 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:47.094697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55300 (local address 127.29.229.3:35371)
0811 22:23:47.100996 (+ 6299us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:47.101005 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:47.101040 (+ 35us) server_negotiation.cc:407] Connection header received
0811 22:23:47.101163 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:47.101170 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:47.101372 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:47.101604 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:47.102335 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.104394 (+ 2059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.108170 (+ 3776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.109737 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.109885 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:47.110122 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:47.115329 (+ 5207us) server_negotiation.cc:299] Negotiation successful
0811 22:23:47.115569 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6013,"thread_start_us":5749,"threads_started":1}
I20260811 22:23:47.118480 28973 tablet_service.cc:1467] Processing CreateTablet for tablet c14fcfa009a24036b9320a1d77452bcf (DEFAULT_TABLE table=test-table [id=4f658aefd651472fa2bb1b54fe6362bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:47.119027 29191 tablet_service.cc:1467] Processing CreateTablet for tablet c14fcfa009a24036b9320a1d77452bcf (DEFAULT_TABLE table=test-table [id=4f658aefd651472fa2bb1b54fe6362bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:47.120211 28973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c14fcfa009a24036b9320a1d77452bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:47.119860 29392 tablet_service.cc:1467] Processing CreateTablet for tablet c14fcfa009a24036b9320a1d77452bcf (DEFAULT_TABLE table=test-table [id=4f658aefd651472fa2bb1b54fe6362bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:47.120520 29191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c14fcfa009a24036b9320a1d77452bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:47.121313 29392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c14fcfa009a24036b9320a1d77452bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:47.136549 29534 tablet_bootstrap.cc:492] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302: Bootstrap starting.
I20260811 22:23:47.141762 29534 tablet_bootstrap.cc:654] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:47.143050 29535 tablet_bootstrap.cc:492] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75: Bootstrap starting.
I20260811 22:23:47.145041 29536 tablet_bootstrap.cc:492] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511: Bootstrap starting.
I20260811 22:23:47.146132 29534 log.cc:824] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:47.147547 29535 tablet_bootstrap.cc:654] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:47.148672 29536 tablet_bootstrap.cc:654] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:47.149608 29534 tablet_bootstrap.cc:492] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302: No bootstrap required, opened a new log
I20260811 22:23:47.149945 29534 ts_tablet_manager.cc:1397] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.002s
I20260811 22:23:47.149927 29535 log.cc:824] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:47.150558 29536 log.cc:824] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:47.156271 29536 tablet_bootstrap.cc:492] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511: No bootstrap required, opened a new log
I20260811 22:23:47.156613 29536 ts_tablet_manager.cc:1397] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20260811 22:23:47.160369 29535 tablet_bootstrap.cc:492] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75: No bootstrap required, opened a new log
I20260811 22:23:47.160681 29535 ts_tablet_manager.cc:1397] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.004s
I20260811 22:23:47.161475 29534 raft_consensus.cc:348] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42541e4a02334d9eacc5d73633059d75" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35371 } } peers { permanent_uuid: "c21bb9ea81e74a89bd385601678c5511" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36029 } } peers { permanent_uuid: "a46afe5a8b06494aab40f25cfdcae302" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40941 } }
I20260811 22:23:47.161875 29534 raft_consensus.cc:374] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:47.161976 29534 raft_consensus.cc:729] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a46afe5a8b06494aab40f25cfdcae302, State: Initialized, Role: FOLLOWER
I20260811 22:23:47.162613 29534 consensus_queue.cc:260] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42541e4a02334d9eacc5d73633059d75" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35371 } } peers { permanent_uuid: "c21bb9ea81e74a89bd385601678c5511" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36029 } } peers { permanent_uuid: "a46afe5a8b06494aab40f25cfdcae302" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40941 } }
I20260811 22:23:47.163875 29286 heartbeater.cc:503] Master 127.29.229.62:37829 was elected leader, sending a full tablet report...
I20260811 22:23:47.164570 29534 ts_tablet_manager.cc:1428] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260811 22:23:47.170693 29536 raft_consensus.cc:348] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42541e4a02334d9eacc5d73633059d75" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35371 } } peers { permanent_uuid: "c21bb9ea81e74a89bd385601678c5511" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36029 } } peers { permanent_uuid: "a46afe5a8b06494aab40f25cfdcae302" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40941 } }
I20260811 22:23:47.171236 29536 raft_consensus.cc:374] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:47.171352 29536 raft_consensus.cc:729] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c21bb9ea81e74a89bd385601678c5511, State: Initialized, Role: FOLLOWER
I20260811 22:23:47.172084 29536 consensus_queue.cc:260] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42541e4a02334d9eacc5d73633059d75" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35371 } } peers { permanent_uuid: "c21bb9ea81e74a89bd385601678c5511" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36029 } } peers { permanent_uuid: "a46afe5a8b06494aab40f25cfdcae302" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40941 } }
I20260811 22:23:47.172392 29535 raft_consensus.cc:348] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42541e4a02334d9eacc5d73633059d75" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35371 } } peers { permanent_uuid: "c21bb9ea81e74a89bd385601678c5511" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36029 } } peers { permanent_uuid: "a46afe5a8b06494aab40f25cfdcae302" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40941 } }
I20260811 22:23:47.172971 29535 raft_consensus.cc:374] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:47.173089 29535 raft_consensus.cc:729] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42541e4a02334d9eacc5d73633059d75, State: Initialized, Role: FOLLOWER
I20260811 22:23:47.174304 29536 ts_tablet_manager.cc:1428] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511: Time spent starting tablet: real 0.018s user 0.010s sys 0.004s
I20260811 22:23:47.179172 29535 consensus_queue.cc:260] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42541e4a02334d9eacc5d73633059d75" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35371 } } peers { permanent_uuid: "c21bb9ea81e74a89bd385601678c5511" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36029 } } peers { permanent_uuid: "a46afe5a8b06494aab40f25cfdcae302" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40941 } }
I20260811 22:23:47.182415 29496 heartbeater.cc:503] Master 127.29.229.62:37829 was elected leader, sending a full tablet report...
I20260811 22:23:47.182886 29535 ts_tablet_manager.cc:1428] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75: Time spent starting tablet: real 0.022s user 0.013s sys 0.003s
W20260811 22:23:47.267040 29497 tablet.cc:2369] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:23:47.275393 29080 tablet.cc:2369] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:23:47.408368 29543 raft_consensus.cc:482] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:23:47.408886 29543 raft_consensus.cc:504] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42541e4a02334d9eacc5d73633059d75" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35371 } } peers { permanent_uuid: "c21bb9ea81e74a89bd385601678c5511" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36029 } } peers { permanent_uuid: "a46afe5a8b06494aab40f25cfdcae302" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40941 } }
I20260811 22:23:47.411154 29543 leader_election.cc:288] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c21bb9ea81e74a89bd385601678c5511 (127.29.229.1:36029), a46afe5a8b06494aab40f25cfdcae302 (127.29.229.2:40941)
W20260811 22:23:47.412882 29288 tablet.cc:2369] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:23:47.419883 29502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:47.411707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:36029 (local address 127.29.229.3:41713)
0811 22:23:47.412022 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:47.412053 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:47.412305 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:47.414029 (+ 1724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:47.414038 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:47.414098 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:47.414523 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:47.414543 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.416426 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.416431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:47.417957 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:47.417970 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.419127 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.419133 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:47.419315 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:47.419339 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:23:47.419447 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 22:23:47.419586 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 22:23:47.419883 29526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:47.411999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:41713 (local address 127.29.229.1:36029)
0811 22:23:47.412270 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:47.412277 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:47.412325 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:23:47.412562 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:47.412570 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:47.412845 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:47.413024 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:47.414774 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.416260 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.418139 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.418950 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.419129 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:47.419207 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:47.419487 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 22:23:47.419643 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260811 22:23:47.421482 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:47.413539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:54641 (local address 127.29.229.2:40941)
0811 22:23:47.413797 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:47.413804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:47.414841 (+ 1037us) server_negotiation.cc:407] Connection header received
0811 22:23:47.415046 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:47.415051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:47.415122 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:47.415225 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:47.415769 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.416693 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.419177 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.420465 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.420651 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:47.420737 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:47.421073 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 22:23:47.421205 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 22:23:47.421528 29544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:47.412589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:40941 (local address 127.29.229.3:54641)
0811 22:23:47.414735 (+ 2146us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:47.414754 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:47.414910 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:47.415248 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:47.415253 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:47.415280 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:47.415603 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:47.415612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.416877 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.416882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:47.418996 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:47.419011 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.420700 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.420708 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:47.420889 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:47.420928 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 22:23:47.421042 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 22:23:47.421169 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1958,"thread_start_us":1571,"threads_started":1}
I20260811 22:23:47.421448 29005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c14fcfa009a24036b9320a1d77452bcf" candidate_uuid: "42541e4a02334d9eacc5d73633059d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c21bb9ea81e74a89bd385601678c5511" is_pre_election: true
I20260811 22:23:47.422549 29005 raft_consensus.cc:2454] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42541e4a02334d9eacc5d73633059d75 in term 0.
I20260811 22:23:47.422744 29222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c14fcfa009a24036b9320a1d77452bcf" candidate_uuid: "42541e4a02334d9eacc5d73633059d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a46afe5a8b06494aab40f25cfdcae302" is_pre_election: true
I20260811 22:23:47.423331 29222 raft_consensus.cc:2454] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42541e4a02334d9eacc5d73633059d75 in term 0.
I20260811 22:23:47.423372 29319 leader_election.cc:302] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42541e4a02334d9eacc5d73633059d75, c21bb9ea81e74a89bd385601678c5511; no voters:
I20260811 22:23:47.424211 29543 raft_consensus.cc:2781] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:23:47.424361 29543 raft_consensus.cc:482] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:23:47.424456 29543 raft_consensus.cc:3037] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:47.427445 29543 raft_consensus.cc:504] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42541e4a02334d9eacc5d73633059d75" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35371 } } peers { permanent_uuid: "c21bb9ea81e74a89bd385601678c5511" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36029 } } peers { permanent_uuid: "a46afe5a8b06494aab40f25cfdcae302" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40941 } }
I20260811 22:23:47.428067 29543 leader_election.cc:288] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [CANDIDATE]: Term 1 election: Requested vote from peers c21bb9ea81e74a89bd385601678c5511 (127.29.229.1:36029), a46afe5a8b06494aab40f25cfdcae302 (127.29.229.2:40941)
I20260811 22:23:47.428715 29005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c14fcfa009a24036b9320a1d77452bcf" candidate_uuid: "42541e4a02334d9eacc5d73633059d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c21bb9ea81e74a89bd385601678c5511"
I20260811 22:23:47.428735 29222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c14fcfa009a24036b9320a1d77452bcf" candidate_uuid: "42541e4a02334d9eacc5d73633059d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a46afe5a8b06494aab40f25cfdcae302"
I20260811 22:23:47.428961 29222 raft_consensus.cc:3037] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:47.428953 29005 raft_consensus.cc:3037] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:47.431226 29005 raft_consensus.cc:2454] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42541e4a02334d9eacc5d73633059d75 in term 1.
I20260811 22:23:47.431231 29222 raft_consensus.cc:2454] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42541e4a02334d9eacc5d73633059d75 in term 1.
I20260811 22:23:47.431660 29319 leader_election.cc:302] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42541e4a02334d9eacc5d73633059d75, a46afe5a8b06494aab40f25cfdcae302; no voters:
I20260811 22:23:47.431977 29543 raft_consensus.cc:2781] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:23:47.432466 29543 raft_consensus.cc:686] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [term 1 LEADER]: Becoming Leader. State: Replica: 42541e4a02334d9eacc5d73633059d75, State: Running, Role: LEADER
I20260811 22:23:47.433045 29543 consensus_queue.cc:237] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42541e4a02334d9eacc5d73633059d75" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35371 } } peers { permanent_uuid: "c21bb9ea81e74a89bd385601678c5511" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36029 } } peers { permanent_uuid: "a46afe5a8b06494aab40f25cfdcae302" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40941 } }
I20260811 22:23:47.437978 28761 catalog_manager.cc:5495] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42541e4a02334d9eacc5d73633059d75 (127.29.229.3). New cstate: current_term: 1 leader_uuid: "42541e4a02334d9eacc5d73633059d75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42541e4a02334d9eacc5d73633059d75" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c21bb9ea81e74a89bd385601678c5511" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a46afe5a8b06494aab40f25cfdcae302" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40941 } health_report { overall_health: UNKNOWN } } }
I20260811 22:23:47.485772 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:47.479574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55314 (local address 127.29.229.3:35371)
0811 22:23:47.479870 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:47.479876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:47.479903 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:23:47.480038 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:47.480042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:47.480162 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:47.480373 (+ 211us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:23:47.480937 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.481731 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.483413 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.483907 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.484073 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:47.484691 (+ 618us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:23:47.485205 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:23:47.485345 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:47.485520 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 22:23:47.485564 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260811 22:23:47.494518 29222 raft_consensus.cc:1261] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302 [term 1 FOLLOWER]: Refusing update from remote peer 42541e4a02334d9eacc5d73633059d75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:23:47.494518 29005 raft_consensus.cc:1261] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511 [term 1 FOLLOWER]: Refusing update from remote peer 42541e4a02334d9eacc5d73633059d75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:23:47.495710 29545 consensus_queue.cc:1035] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c21bb9ea81e74a89bd385601678c5511" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 36029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:23:47.496155 29543 consensus_queue.cc:1035] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a46afe5a8b06494aab40f25cfdcae302" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 40941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:23:47.527768 28761 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:23:47.528020 28761 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:23:47.531976 28761 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e35a3f3ebff0421dab00c2a597eef1ae: Sending DeleteTablet for 3 replicas of tablet c14fcfa009a24036b9320a1d77452bcf
I20260811 22:23:47.533300 29392 tablet_service.cc:1514] Processing DeleteTablet for tablet c14fcfa009a24036b9320a1d77452bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:47 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:55300
I20260811 22:23:47.533460 28973 tablet_service.cc:1514] Processing DeleteTablet for tablet c14fcfa009a24036b9320a1d77452bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:47 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:44366
I20260811 22:23:47.533566 29191 tablet_service.cc:1514] Processing DeleteTablet for tablet c14fcfa009a24036b9320a1d77452bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:47 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:38824
I20260811 22:23:47.539790 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 28871
I20260811 22:23:47.553205 29569 tablet_replica.cc:330] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302: stopping tablet replica
I20260811 22:23:47.553491 29568 tablet_replica.cc:330] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511: stopping tablet replica
I20260811 22:23:47.553742 29569 raft_consensus.cc:2229] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:23:47.553961 29568 raft_consensus.cc:2229] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:23:47.554327 29569 raft_consensus.cc:2258] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:47.554525 29568 raft_consensus.cc:2258] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:47.558812 29566 tablet_replica.cc:330] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75: stopping tablet replica
I20260811 22:23:47.558987 29568 ts_tablet_manager.cc:1905] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:47.559286 29566 raft_consensus.cc:2229] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:23:47.560063 29566 raft_consensus.cc:2258] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:47.563361 29566 ts_tablet_manager.cc:1905] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:47.565727 29568 ts_tablet_manager.cc:1918] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:23:47.565913 29568 log.cc:1196] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-0/wal/wals/c14fcfa009a24036b9320a1d77452bcf
I20260811 22:23:47.566502 29568 ts_tablet_manager.cc:1939] T c14fcfa009a24036b9320a1d77452bcf P c21bb9ea81e74a89bd385601678c5511: Deleting consensus metadata
I20260811 22:23:47.566473 29569 ts_tablet_manager.cc:1905] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:47.568504 28736 catalog_manager.cc:4841] TS c21bb9ea81e74a89bd385601678c5511 (127.29.229.1:36029): tablet c14fcfa009a24036b9320a1d77452bcf (table test-table [id=4f658aefd651472fa2bb1b54fe6362bc]) successfully deleted
I20260811 22:23:47.571776 29566 ts_tablet_manager.cc:1918] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:23:47.572067 29566 log.cc:1196] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/c14fcfa009a24036b9320a1d77452bcf
Aug 11 22:23:47 dist-test-slave-jq4t krb5kdc[28693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:23:47.572781 29566 ts_tablet_manager.cc:1939] T c14fcfa009a24036b9320a1d77452bcf P 42541e4a02334d9eacc5d73633059d75: Deleting consensus metadata
I20260811 22:23:47.573642 29569 ts_tablet_manager.cc:1918] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:23:47.573807 29569 log.cc:1196] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/c14fcfa009a24036b9320a1d77452bcf
I20260811 22:23:47.574373 29569 ts_tablet_manager.cc:1939] T c14fcfa009a24036b9320a1d77452bcf P a46afe5a8b06494aab40f25cfdcae302: Deleting consensus metadata
I20260811 22:23:47.575071 28734 catalog_manager.cc:4841] TS 42541e4a02334d9eacc5d73633059d75 (127.29.229.3:35371): tablet c14fcfa009a24036b9320a1d77452bcf (table test-table [id=4f658aefd651472fa2bb1b54fe6362bc]) successfully deleted
I20260811 22:23:47.576128 28736 catalog_manager.cc:4841] TS a46afe5a8b06494aab40f25cfdcae302 (127.29.229.2:40941): tablet c14fcfa009a24036b9320a1d77452bcf (table test-table [id=4f658aefd651472fa2bb1b54fe6362bc]) successfully deleted
I20260811 22:23:47.580394 29526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:47.541988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44376 (local address 127.29.229.1:36029)
0811 22:23:47.542244 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:47.542253 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:47.555774 (+ 13521us) server_negotiation.cc:407] Connection header received
0811 22:23:47.556098 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:47.556104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:47.556193 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:47.556271 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:47.560526 (+ 4255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.561740 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.564213 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.564912 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.565029 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:47.574149 (+ 9120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:47.574165 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:47.574184 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:47.574240 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:47.577516 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:47.578233 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:47.578246 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:47.578249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:47.578384 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:47.578791 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:47.578796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:47.578801 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:47.579160 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:47.579487 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:47.579806 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 22:23:47.580027 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:23:47.581146 29072 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:47.693051 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 28871
I20260811 22:23:47.722465 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 29086
Aug 11 22:23:47 dist-test-slave-jq4t krb5kdc[28693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.2@KRBTEST.COM
I20260811 22:23:47.740869 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:47.723427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38840 (local address 127.29.229.2:40941)
0811 22:23:47.723650 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:47.723656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:47.723676 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:23:47.723821 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:47.723826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:47.723904 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:47.724032 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:47.725525 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.726660 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.727956 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.728777 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.728911 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:47.733699 (+ 4788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:47.733727 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:47.733752 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:47.733802 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:47.737756 (+ 3954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:47.738550 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:47.738560 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:47.738564 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:47.738716 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:47.739082 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:47.739087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:47.739091 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:47.739587 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:47.739905 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:47.740209 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 22:23:47.740446 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:23:47.741678 29279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:47.836951 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 29086
I20260811 22:23:47.873701 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 29294
Aug 11 22:23:47 dist-test-slave-jq4t krb5kdc[28693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.3@KRBTEST.COM
I20260811 22:23:47.893965 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:47.874592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55318 (local address 127.29.229.3:35371)
0811 22:23:47.874776 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:47.874781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:47.874864 (+ 83us) server_negotiation.cc:407] Connection header received
0811 22:23:47.875078 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:47.875082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:47.875147 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:47.875235 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:47.877224 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.878256 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.880230 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:47.880935 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:47.881051 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:47.886998 (+ 5947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:47.887017 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:47.887037 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:47.887086 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:47.890760 (+ 3674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:47.891542 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:47.891555 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:47.891557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:47.891731 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:47.892159 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:47.892165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:47.892173 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:47.892636 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:47.892979 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:47.893282 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 22:23:47.893545 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:23:47.894712 29488 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:48.028996 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 29294
I20260811 22:23:48.066507 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 28712
I20260811 22:23:48.067694 28817 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:48.182382 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 28712
2026-08-11T22:23:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5448 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T22:23:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:23:48Z Disabled control of system clock
I20260811 22:23:48.297297 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:39497
--webserver_interface=127.29.229.62
--webserver_port=0
--builtin_ntp_servers=127.29.229.20:46473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:39497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:23:48.664451 29588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:48.664876 29588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:48.665019 29588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:48.673099 29588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:23:48.673231 29588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:48.673272 29588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:48.673369 29588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:23:48.673437 29588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:23:48.684008 29588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:39497
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:39497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:48.685458 29588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:48.687402 29588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:48.701113 29598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:48.701030 29603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:48.701109 29599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:48.702013 29588 server_base.cc:1034] running on GCE node
I20260811 22:23:48.702862 29588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:48.704417 29588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:48.705694 29588 hybrid_clock.cc:648] HybridClock initialized: now 1786487028705635 us; error 51 us; skew 500 ppm
I20260811 22:23:48.706102 29588 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:23:48.708793 29588 webserver.cc:460] Webserver started at http://127.29.229.62:40349/ using document root <none> and password file <none>
I20260811 22:23:48.709532 29588 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:48.709671 29588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:48.710042 29588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:48.713178 29588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "462e3922654b457fb92a43fb5feee01e"
format_stamp: "Formatted at 2026-08-11 22:23:48 on dist-test-slave-jq4t"
server_key: "4fd5b110e19f8cc97fbbadba27d54229"
server_key_iv: "54e41c016a0d0f086688a478849cb1c7"
server_key_version: "encryptionkey@0"
I20260811 22:23:48.714195 29588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "462e3922654b457fb92a43fb5feee01e"
format_stamp: "Formatted at 2026-08-11 22:23:48 on dist-test-slave-jq4t"
server_key: "4fd5b110e19f8cc97fbbadba27d54229"
server_key_iv: "54e41c016a0d0f086688a478849cb1c7"
server_key_version: "encryptionkey@0"
I20260811 22:23:48.719695 29588 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 22:23:48.723346 29614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:48.724843 29588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:23:48.725050 29588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "462e3922654b457fb92a43fb5feee01e"
format_stamp: "Formatted at 2026-08-11 22:23:48 on dist-test-slave-jq4t"
server_key: "4fd5b110e19f8cc97fbbadba27d54229"
server_key_iv: "54e41c016a0d0f086688a478849cb1c7"
server_key_version: "encryptionkey@0"
I20260811 22:23:48.725288 29588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:48.758230 29588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:48.759193 29588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:48.759533 29588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:48.781944 29588 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:39497
I20260811 22:23:48.781963 29696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:39497 every 8 connection(s)
I20260811 22:23:48.783435 29588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:23:48.784188 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 29588
I20260811 22:23:48.784504 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:23:48.785149 30612 external_mini_cluster.cc:1428] Setting key 65ff9b3acbb5a6e3559187900dff6803
I20260811 22:23:48.788172 29698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:48.800302 29703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:48.787216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42836 (local address 127.29.229.62:39497)
0811 22:23:48.788215 (+ 999us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:48.788253 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:48.788314 (+ 61us) server_negotiation.cc:407] Connection header received
0811 22:23:48.790083 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:48.790104 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:48.790573 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:48.790933 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:48.791611 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:48.793099 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:48.795306 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:48.795993 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:48.796140 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:48.797609 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:48.797616 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:48.797630 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:48.798399 (+ 769us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:48.798481 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:48.798507 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:48.798604 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:48.798879 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 22:23:48.799270 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":384,"threads_started":1}
I20260811 22:23:48.801757 29698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e: Bootstrap starting.
I20260811 22:23:48.805665 29698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:48.806981 29698 log.cc:824] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:48.809827 29698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e: No bootstrap required, opened a new log
I20260811 22:23:48.818296 29698 raft_consensus.cc:348] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e3922654b457fb92a43fb5feee01e" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 39497 } }
I20260811 22:23:48.818699 29698 raft_consensus.cc:374] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:48.818847 29698 raft_consensus.cc:729] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 462e3922654b457fb92a43fb5feee01e, State: Initialized, Role: FOLLOWER
I20260811 22:23:48.819382 29698 consensus_queue.cc:260] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e3922654b457fb92a43fb5feee01e" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 39497 } }
I20260811 22:23:48.819602 29698 raft_consensus.cc:388] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:23:48.819705 29698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:23:48.819846 29698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:48.822767 29698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e3922654b457fb92a43fb5feee01e" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 39497 } }
I20260811 22:23:48.823313 29698 leader_election.cc:302] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 462e3922654b457fb92a43fb5feee01e; no voters:
I20260811 22:23:48.823876 29698 leader_election.cc:288] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:23:48.824055 29709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:23:48.824877 29709 raft_consensus.cc:686] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e [term 1 LEADER]: Becoming Leader. State: Replica: 462e3922654b457fb92a43fb5feee01e, State: Running, Role: LEADER
I20260811 22:23:48.825414 29709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e3922654b457fb92a43fb5feee01e" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 39497 } }
I20260811 22:23:48.826280 29698 sys_catalog.cc:564] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:23:48.828457 29713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 462e3922654b457fb92a43fb5feee01e. Latest consensus state: current_term: 1 leader_uuid: "462e3922654b457fb92a43fb5feee01e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e3922654b457fb92a43fb5feee01e" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 39497 } } }
I20260811 22:23:48.829167 29713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:48.828466 29710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "462e3922654b457fb92a43fb5feee01e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e3922654b457fb92a43fb5feee01e" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 39497 } } }
I20260811 22:23:48.829807 29710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:48.832012 29719 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:23:48.837893 29719 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:23:48.849699 29719 catalog_manager.cc:1347] Generated new cluster ID: 5becfcd46c3f49109ed7cfe73e9e947d
I20260811 22:23:48.849870 29719 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:23:48.860000 29719 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:23:48.861469 29719 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:23:48.869714 29719 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e: Generated new TSK 0
I20260811 22:23:48.870720 29719 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:23:48.876271 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:39497
--builtin_ntp_servers=127.29.229.20:46473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:23:49.255924 29740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:49.256217 29740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:49.256367 29740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:49.264462 29740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:49.264627 29740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:49.264823 29740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:23:49.274163 29740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:39497
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:49.275539 29740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:49.277479 29740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:49.293331 29747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:49.293244 29753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:49.293324 29748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:49.297597 29740 server_base.cc:1034] running on GCE node
I20260811 22:23:49.298338 29740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:49.299508 29740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:49.300778 29740 hybrid_clock.cc:648] HybridClock initialized: now 1786487029300700 us; error 103 us; skew 500 ppm
I20260811 22:23:49.301172 29740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:23:49.303709 29740 webserver.cc:460] Webserver started at http://127.29.229.1:43093/ using document root <none> and password file <none>
I20260811 22:23:49.304579 29740 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:49.304718 29740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:49.305083 29740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:49.308297 29740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "cf71a0b394054dc2bcb028edf53b53c8"
format_stamp: "Formatted at 2026-08-11 22:23:49 on dist-test-slave-jq4t"
server_key: "22536c6c00f4c70d29335a7a3b9969a2"
server_key_iv: "277d74050c794beb77323f42baef6740"
server_key_version: "encryptionkey@0"
I20260811 22:23:49.309226 29740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "cf71a0b394054dc2bcb028edf53b53c8"
format_stamp: "Formatted at 2026-08-11 22:23:49 on dist-test-slave-jq4t"
server_key: "22536c6c00f4c70d29335a7a3b9969a2"
server_key_iv: "277d74050c794beb77323f42baef6740"
server_key_version: "encryptionkey@0"
I20260811 22:23:49.314675 29740 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 22:23:49.318356 29761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:49.319730 29740 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:23:49.319895 29740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "cf71a0b394054dc2bcb028edf53b53c8"
format_stamp: "Formatted at 2026-08-11 22:23:49 on dist-test-slave-jq4t"
server_key: "22536c6c00f4c70d29335a7a3b9969a2"
server_key_iv: "277d74050c794beb77323f42baef6740"
server_key_version: "encryptionkey@0"
I20260811 22:23:49.320150 29740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:49.338737 29740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:49.339622 29740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:49.339946 29740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:49.341040 29740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:49.342602 29740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:49.342696 29740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:49.342777 29740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:49.342839 29740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:49.388442 29740 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:39665
I20260811 22:23:49.388517 29950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:39665 every 8 connection(s)
I20260811 22:23:49.390379 29740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:23:49.398528 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 29740
I20260811 22:23:49.398733 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:23:49.399214 30612 external_mini_cluster.cc:1428] Setting key 087946462adeed270319705011b34388
I20260811 22:23:49.402035 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:39497
--builtin_ntp_servers=127.29.229.20:46473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:23:49.404779 29955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:49.393713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:46819 (local address 127.29.229.62:39497)
0811 22:23:49.394270 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:49.394277 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:49.395796 (+ 1519us) server_negotiation.cc:407] Connection header received
0811 22:23:49.397718 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:49.397724 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:49.397813 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:49.397948 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:49.399163 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:49.399967 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:49.401692 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:49.402115 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:49.402302 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:49.403837 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:49.403843 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:49.403850 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:49.403875 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:49.403916 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:49.403928 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:49.404027 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:49.404268 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 22:23:49.404381 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":285,"threads_started":1}
I20260811 22:23:49.405809 29956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:49.394421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:39497 (local address 127.29.229.1:46819)
0811 22:23:49.395644 (+ 1223us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:49.395685 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:49.397446 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:49.398197 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:49.398206 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:49.398292 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:49.398995 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:49.399015 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:49.400133 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:49.400139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:49.401536 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:49.401549 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:49.402506 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:49.402512 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:49.402726 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:49.403497 (+ 771us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:23:49.403500 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:49.403542 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:23:49.403630 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:49.404029 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:49.404042 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:49.404057 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 22:23:49.404256 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 22:23:49.404702 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":338,"threads_started":1}
I20260811 22:23:49.408097 29952 heartbeater.cc:347] Connected to a master server at 127.29.229.62:39497
I20260811 22:23:49.408463 29952 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:49.409363 29952 heartbeater.cc:511] Master 127.29.229.62:39497 requested a full tablet report, sending...
I20260811 22:23:49.411837 29641 ts_manager.cc:194] Registered new tserver with Master: cf71a0b394054dc2bcb028edf53b53c8 (127.29.229.1:39665)
I20260811 22:23:49.414105 29641 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.229.1:46819
W20260811 22:23:49.783766 29957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:49.784363 29957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:49.784637 29957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:49.793970 29957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:49.794113 29957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:49.794304 29957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:23:49.804605 29957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:39497
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:49.806025 29957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:49.807961 29957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:49.823243 29965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:49.823559 29966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:49.824069 29968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:49.825099 29957 server_base.cc:1034] running on GCE node
I20260811 22:23:49.825863 29957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:49.827255 29957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:49.828594 29957 hybrid_clock.cc:648] HybridClock initialized: now 1786487029828513 us; error 53 us; skew 500 ppm
I20260811 22:23:49.829015 29957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:23:49.831992 29957 webserver.cc:460] Webserver started at http://127.29.229.2:44289/ using document root <none> and password file <none>
I20260811 22:23:49.832937 29957 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:49.833082 29957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:49.833479 29957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:49.837180 29957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "d7be74bc8cb54f47b3210053132a1fa6"
format_stamp: "Formatted at 2026-08-11 22:23:49 on dist-test-slave-jq4t"
server_key: "7ccc5d8f0fb516a41bf4127f600d83d4"
server_key_iv: "281e4300c5e9ddae7aa514815ba74f69"
server_key_version: "encryptionkey@0"
I20260811 22:23:49.838106 29957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "d7be74bc8cb54f47b3210053132a1fa6"
format_stamp: "Formatted at 2026-08-11 22:23:49 on dist-test-slave-jq4t"
server_key: "7ccc5d8f0fb516a41bf4127f600d83d4"
server_key_iv: "281e4300c5e9ddae7aa514815ba74f69"
server_key_version: "encryptionkey@0"
I20260811 22:23:49.843808 29957 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:23:49.847600 29985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:49.849267 29957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:23:49.849479 29957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "d7be74bc8cb54f47b3210053132a1fa6"
format_stamp: "Formatted at 2026-08-11 22:23:49 on dist-test-slave-jq4t"
server_key: "7ccc5d8f0fb516a41bf4127f600d83d4"
server_key_iv: "281e4300c5e9ddae7aa514815ba74f69"
server_key_version: "encryptionkey@0"
I20260811 22:23:49.849723 29957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:49.899505 29957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:49.900947 29957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:49.901355 29957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:49.902585 29957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:49.904331 29957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:49.904466 29957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:49.904647 29957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:49.904742 29957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:49.944351 29957 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:45987
I20260811 22:23:49.944358 30177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:45987 every 8 connection(s)
I20260811 22:23:49.945812 29957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:23:49.946357 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 29957
I20260811 22:23:49.946575 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:23:49.947232 30612 external_mini_cluster.cc:1428] Setting key 56e677a5259f3c8e31de38554a27a9fe
I20260811 22:23:49.950774 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:39497
--builtin_ntp_servers=127.29.229.20:46473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:23:49.963120 30188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:49.948724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:37517 (local address 127.29.229.62:39497)
0811 22:23:49.949223 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:49.949229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:49.950840 (+ 1611us) server_negotiation.cc:407] Connection header received
0811 22:23:49.953945 (+ 3105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:49.953950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:49.954039 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:49.954148 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:49.955745 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:49.956690 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:49.959292 (+ 2602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:49.959727 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:49.959916 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:49.962193 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:49.962198 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:49.962202 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:49.962224 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:49.962270 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:49.962283 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:49.962367 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:49.962722 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 22:23:49.962827 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":244,"threads_started":1}
I20260811 22:23:49.964751 30190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:49.949511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:39497 (local address 127.29.229.2:37517)
0811 22:23:49.950618 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:49.950679 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:49.953589 (+ 2910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:49.954437 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:49.954451 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:49.954576 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:49.955476 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:49.955512 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:49.956860 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:49.956869 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:49.959105 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:49.959125 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:49.959989 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:49.960001 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:49.960356 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:49.961729 (+ 1373us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:23:49.961736 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:49.961808 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:23:49.961938 (+ 130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:49.962420 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:49.962436 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:49.962449 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 22:23:49.962681 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 22:23:49.963155 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":319,"threads_started":1}
I20260811 22:23:49.966604 30180 heartbeater.cc:347] Connected to a master server at 127.29.229.62:39497
I20260811 22:23:49.966962 30180 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:49.967818 30180 heartbeater.cc:511] Master 127.29.229.62:39497 requested a full tablet report, sending...
I20260811 22:23:49.969619 29641 ts_manager.cc:194] Registered new tserver with Master: d7be74bc8cb54f47b3210053132a1fa6 (127.29.229.2:45987)
I20260811 22:23:49.970507 29641 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.229.2:37517
W20260811 22:23:50.334424 30192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:50.334731 30192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:50.334875 30192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:50.342603 30192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:50.342727 30192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:50.342932 30192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:23:50.352594 30192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:46473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:39497
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:50.353994 30192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:50.356098 30192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:50.371456 30203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:50.371565 30202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:50.371547 30205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:50.372442 30192 server_base.cc:1034] running on GCE node
I20260811 22:23:50.373286 30192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:50.375003 30192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:50.376362 30192 hybrid_clock.cc:648] HybridClock initialized: now 1786487030376285 us; error 66 us; skew 500 ppm
I20260811 22:23:50.376973 30192 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:23:50.383893 30192 webserver.cc:460] Webserver started at http://127.29.229.3:36525/ using document root <none> and password file <none>
I20260811 22:23:50.385145 30192 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:50.385339 30192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:50.385792 30192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:50.389832 30192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "a5f55973c7a3410ca1debebdf0cb707b"
format_stamp: "Formatted at 2026-08-11 22:23:50 on dist-test-slave-jq4t"
server_key: "2ff6caedc00b3e2c0e037fad0cdee773"
server_key_iv: "3b3fb348b5d0f3cd5354eef6138cdbed"
server_key_version: "encryptionkey@0"
I20260811 22:23:50.390956 30192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "a5f55973c7a3410ca1debebdf0cb707b"
format_stamp: "Formatted at 2026-08-11 22:23:50 on dist-test-slave-jq4t"
server_key: "2ff6caedc00b3e2c0e037fad0cdee773"
server_key_iv: "3b3fb348b5d0f3cd5354eef6138cdbed"
server_key_version: "encryptionkey@0"
I20260811 22:23:50.398811 30192 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260811 22:23:50.403491 30217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:50.405226 30192 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 22:23:50.405452 30192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "a5f55973c7a3410ca1debebdf0cb707b"
format_stamp: "Formatted at 2026-08-11 22:23:50 on dist-test-slave-jq4t"
server_key: "2ff6caedc00b3e2c0e037fad0cdee773"
server_key_iv: "3b3fb348b5d0f3cd5354eef6138cdbed"
server_key_version: "encryptionkey@0"
I20260811 22:23:50.405714 30192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:50.417871 29952 heartbeater.cc:503] Master 127.29.229.62:39497 was elected leader, sending a full tablet report...
I20260811 22:23:50.433614 30192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:50.434525 30192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:50.434851 30192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:50.436008 30192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:50.437709 30192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:50.437837 30192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:50.437978 30192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:50.438078 30192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:50.477783 30192 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:36373
I20260811 22:23:50.477875 30415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:36373 every 8 connection(s)
I20260811 22:23:50.479224 30192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:23:50.486193 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 30192
I20260811 22:23:50.486420 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:23:50.486923 30612 external_mini_cluster.cc:1428] Setting key 05dce0c7ea2114062429558726f4cd59
I20260811 22:23:50.494534 30418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:50.482783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:35837 (local address 127.29.229.62:39497)
0811 22:23:50.483265 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:50.483270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:50.484520 (+ 1250us) server_negotiation.cc:407] Connection header received
0811 22:23:50.486611 (+ 2091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:50.486618 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:50.486721 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:50.486853 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:50.488238 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.489331 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.491507 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.491931 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.492037 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:50.493705 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:50.493710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:50.493713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:50.493732 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:50.493774 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:50.493787 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:50.493870 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:50.494154 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 22:23:50.494238 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":261,"threads_started":1}
I20260811 22:23:50.495678 30419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:50.483264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:39497 (local address 127.29.229.3:35837)
0811 22:23:50.484313 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:50.484350 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:50.486249 (+ 1899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:50.487131 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:50.487141 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:50.487296 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:50.488035 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:50.488065 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.489560 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.489565 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:50.491315 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:50.491337 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.492171 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.492177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:50.492391 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:50.493359 (+ 968us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:23:50.493362 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:50.493405 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:23:50.493505 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:50.493929 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:50.493939 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:50.493946 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:23:50.494118 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 22:23:50.494447 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":357,"threads_started":1}
I20260811 22:23:50.497267 30416 heartbeater.cc:347] Connected to a master server at 127.29.229.62:39497
I20260811 22:23:50.497609 30416 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:50.498438 30416 heartbeater.cc:511] Master 127.29.229.62:39497 requested a full tablet report, sending...
I20260811 22:23:50.500085 29641 ts_manager.cc:194] Registered new tserver with Master: a5f55973c7a3410ca1debebdf0cb707b (127.29.229.3:36373)
I20260811 22:23:50.500982 29641 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.229.3:35837
I20260811 22:23:50.504029 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:23:50.514647 30418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:50.508508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42844 (local address 127.29.229.62:39497)
0811 22:23:50.508679 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:50.508684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:50.509093 (+ 409us) server_negotiation.cc:407] Connection header received
0811 22:23:50.509226 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:50.509230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:50.509329 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:50.509416 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:50.509935 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.510695 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.512152 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.512672 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.512778 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:50.513936 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:50.513942 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:50.513945 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:50.513962 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:50.513995 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:50.514009 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:50.514124 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:50.514342 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 22:23:50.514392 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:23:50.518967 29641 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:23:50.526156 29641 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:23:50.553288 30442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:50.540638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:39665 (local address 127.0.0.1:44156)
0811 22:23:50.541737 (+ 1099us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:50.541799 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:50.542045 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:50.543980 (+ 1935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:50.543987 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:50.544028 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:50.544572 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:50.544589 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.548145 (+ 3556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.548152 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:50.550604 (+ 2452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:50.550618 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.552296 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.552303 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:50.552532 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:50.552598 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 22:23:50.552719 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 22:23:50.552924 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":1103,"threads_started":1}
I20260811 22:23:50.553668 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:50.541025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44156 (local address 127.29.229.1:39665)
0811 22:23:50.543269 (+ 2244us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:50.543280 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:50.543312 (+ 32us) server_negotiation.cc:407] Connection header received
0811 22:23:50.543433 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:50.543440 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:50.543717 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:50.544776 (+ 1059us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:50.545110 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.547391 (+ 2281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.550848 (+ 3457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.552087 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.552239 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:50.552584 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:50.552968 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 22:23:50.553148 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2075,"thread_start_us":1950,"threads_started":1}
I20260811 22:23:50.558283 30446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:50.541493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:45987 (local address 127.0.0.1:51216)
0811 22:23:50.543598 (+ 2105us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:50.543617 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:50.543829 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:50.544760 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:50.544764 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:50.544792 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:50.551081 (+ 6289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:50.551089 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.553258 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.553263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:50.555783 (+ 2520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:50.555796 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.557448 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.557455 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:50.557656 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:50.557689 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 22:23:50.557794 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 22:23:50.557929 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1988,"mutex_wait_us":82,"thread_start_us":6855,"threads_started":1}
I20260811 22:23:50.558638 30445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:50.542108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51216 (local address 127.29.229.2:45987)
0811 22:23:50.542787 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:50.542800 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:50.543715 (+ 915us) server_negotiation.cc:407] Connection header received
0811 22:23:50.544040 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:50.544048 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:50.544254 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:50.544453 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:50.551296 (+ 6843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.553032 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.555978 (+ 2946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.557238 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.557387 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:50.557628 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:50.557861 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 22:23:50.558030 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":300,"threads_started":1}
I20260811 22:23:50.558334 29844 tablet_service.cc:1467] Processing CreateTablet for tablet 665aca4a5ac347b994c115b406576918 (DEFAULT_TABLE table=test-table [id=2c9e4bb93eaf4136a27b462332618dd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:50.559681 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:50.542616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:36373 (local address 127.0.0.1:59858)
0811 22:23:50.548522 (+ 5906us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:50.548552 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:50.548684 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:50.549742 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:50.549747 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:50.549777 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:50.551285 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:50.551299 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.554139 (+ 2840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.554145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:50.556676 (+ 2531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:50.556692 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.558847 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.558853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:50.559066 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:50.559103 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 22:23:50.559194 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:23:50.559329 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5738,"mutex_wait_us":3,"thread_start_us":1081,"threads_started":1}
I20260811 22:23:50.560443 30448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:50.543357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59858 (local address 127.29.229.3:36373)
0811 22:23:50.548076 (+ 4719us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:50.548097 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:50.548828 (+ 731us) server_negotiation.cc:407] Connection header received
0811 22:23:50.548939 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:50.548957 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:50.549189 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:50.549393 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:50.551488 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.553084 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.556892 (+ 3808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.558308 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.558454 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:50.558719 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:50.559988 (+ 1269us) server_negotiation.cc:299] Negotiation successful
0811 22:23:50.560161 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4528,"thread_start_us":4228,"threads_started":1}
I20260811 22:23:50.560534 29844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 665aca4a5ac347b994c115b406576918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:50.562835 30077 tablet_service.cc:1467] Processing CreateTablet for tablet 665aca4a5ac347b994c115b406576918 (DEFAULT_TABLE table=test-table [id=2c9e4bb93eaf4136a27b462332618dd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:50.563084 30297 tablet_service.cc:1467] Processing CreateTablet for tablet 665aca4a5ac347b994c115b406576918 (DEFAULT_TABLE table=test-table [id=2c9e4bb93eaf4136a27b462332618dd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:50.564291 30077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 665aca4a5ac347b994c115b406576918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:50.564558 30297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 665aca4a5ac347b994c115b406576918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:50.575680 30449 tablet_bootstrap.cc:492] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b: Bootstrap starting.
I20260811 22:23:50.579596 30454 tablet_bootstrap.cc:492] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8: Bootstrap starting.
I20260811 22:23:50.580672 30449 tablet_bootstrap.cc:654] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:50.581321 30455 tablet_bootstrap.cc:492] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6: Bootstrap starting.
I20260811 22:23:50.583374 30454 tablet_bootstrap.cc:654] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:50.583438 30449 log.cc:824] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:50.584618 30455 tablet_bootstrap.cc:654] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:50.584846 30454 log.cc:824] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:50.585686 30455 log.cc:824] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:50.588435 30449 tablet_bootstrap.cc:492] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b: No bootstrap required, opened a new log
I20260811 22:23:50.588434 30454 tablet_bootstrap.cc:492] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8: No bootstrap required, opened a new log
I20260811 22:23:50.588579 30455 tablet_bootstrap.cc:492] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6: No bootstrap required, opened a new log
I20260811 22:23:50.588773 30449 ts_tablet_manager.cc:1397] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260811 22:23:50.588765 30454 ts_tablet_manager.cc:1397] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260811 22:23:50.588842 30455 ts_tablet_manager.cc:1397] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260811 22:23:50.596123 30455 raft_consensus.cc:348] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf71a0b394054dc2bcb028edf53b53c8" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39665 } } peers { permanent_uuid: "a5f55973c7a3410ca1debebdf0cb707b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36373 } } peers { permanent_uuid: "d7be74bc8cb54f47b3210053132a1fa6" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 45987 } }
I20260811 22:23:50.596568 30455 raft_consensus.cc:374] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:50.596668 30455 raft_consensus.cc:729] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7be74bc8cb54f47b3210053132a1fa6, State: Initialized, Role: FOLLOWER
I20260811 22:23:50.597348 30455 consensus_queue.cc:260] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf71a0b394054dc2bcb028edf53b53c8" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39665 } } peers { permanent_uuid: "a5f55973c7a3410ca1debebdf0cb707b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36373 } } peers { permanent_uuid: "d7be74bc8cb54f47b3210053132a1fa6" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 45987 } }
I20260811 22:23:50.598537 30180 heartbeater.cc:503] Master 127.29.229.62:39497 was elected leader, sending a full tablet report...
I20260811 22:23:50.599263 30455 ts_tablet_manager.cc:1428] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6: Time spent starting tablet: real 0.010s user 0.009s sys 0.001s
I20260811 22:23:50.599845 30454 raft_consensus.cc:348] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf71a0b394054dc2bcb028edf53b53c8" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39665 } } peers { permanent_uuid: "a5f55973c7a3410ca1debebdf0cb707b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36373 } } peers { permanent_uuid: "d7be74bc8cb54f47b3210053132a1fa6" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 45987 } }
I20260811 22:23:50.599844 30449 raft_consensus.cc:348] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf71a0b394054dc2bcb028edf53b53c8" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39665 } } peers { permanent_uuid: "a5f55973c7a3410ca1debebdf0cb707b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36373 } } peers { permanent_uuid: "d7be74bc8cb54f47b3210053132a1fa6" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 45987 } }
I20260811 22:23:50.600307 30449 raft_consensus.cc:374] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:50.600301 30454 raft_consensus.cc:374] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:50.600400 30449 raft_consensus.cc:729] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5f55973c7a3410ca1debebdf0cb707b, State: Initialized, Role: FOLLOWER
I20260811 22:23:50.600399 30454 raft_consensus.cc:729] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf71a0b394054dc2bcb028edf53b53c8, State: Initialized, Role: FOLLOWER
I20260811 22:23:50.601194 30454 consensus_queue.cc:260] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf71a0b394054dc2bcb028edf53b53c8" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39665 } } peers { permanent_uuid: "a5f55973c7a3410ca1debebdf0cb707b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36373 } } peers { permanent_uuid: "d7be74bc8cb54f47b3210053132a1fa6" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 45987 } }
I20260811 22:23:50.601194 30449 consensus_queue.cc:260] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf71a0b394054dc2bcb028edf53b53c8" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39665 } } peers { permanent_uuid: "a5f55973c7a3410ca1debebdf0cb707b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36373 } } peers { permanent_uuid: "d7be74bc8cb54f47b3210053132a1fa6" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 45987 } }
I20260811 22:23:50.602833 30416 heartbeater.cc:503] Master 127.29.229.62:39497 was elected leader, sending a full tablet report...
I20260811 22:23:50.603155 30454 ts_tablet_manager.cc:1428] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8: Time spent starting tablet: real 0.014s user 0.008s sys 0.006s
I20260811 22:23:50.604049 30449 ts_tablet_manager.cc:1428] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
W20260811 22:23:50.643342 29953 tablet.cc:2369] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:23:50.667673 30464 raft_consensus.cc:482] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:23:50.668084 30464 raft_consensus.cc:504] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf71a0b394054dc2bcb028edf53b53c8" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39665 } } peers { permanent_uuid: "a5f55973c7a3410ca1debebdf0cb707b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36373 } } peers { permanent_uuid: "d7be74bc8cb54f47b3210053132a1fa6" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 45987 } }
I20260811 22:23:50.669914 30464 leader_election.cc:288] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5f55973c7a3410ca1debebdf0cb707b (127.29.229.3:36373), d7be74bc8cb54f47b3210053132a1fa6 (127.29.229.2:45987)
I20260811 22:23:50.676335 30445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:50.670940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:42923 (local address 127.29.229.2:45987)
0811 22:23:50.671141 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:50.671146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:50.671168 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:23:50.671286 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:50.671298 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:50.671368 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:50.671482 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:50.672012 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.672797 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.674666 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.675413 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.675604 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:50.675676 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:50.675945 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 22:23:50.676072 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:23:50.676386 30467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:50.670189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:45987 (local address 127.29.229.1:42923)
0811 22:23:50.670947 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:50.670968 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:50.671129 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:50.671502 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:50.671507 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:50.671533 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:50.671860 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:50.671869 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.672938 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.672945 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:50.674466 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:50.674478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.675610 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.675616 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:50.675774 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:50.675798 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:23:50.675904 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 22:23:50.676050 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":379,"threads_started":1}
I20260811 22:23:50.676692 30448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:50.670516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:39747 (local address 127.29.229.3:36373)
0811 22:23:50.670780 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:50.670789 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:50.671503 (+ 714us) server_negotiation.cc:407] Connection header received
0811 22:23:50.671741 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:50.671746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:50.671804 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:50.671904 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:50.672446 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.673207 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.674957 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.675749 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.675936 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:50.676013 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:50.676293 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 22:23:50.676419 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 22:23:50.677138 30469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:50.670303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:36373 (local address 127.29.229.1:39747)
0811 22:23:50.671387 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:50.671401 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:50.671582 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:50.671920 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:50.671924 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:50.671947 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:50.672291 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:50.672300 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.673368 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.673373 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:50.674759 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:50.674771 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.675959 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.675967 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:50.676134 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:50.676157 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:23:50.676261 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 22:23:50.676387 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":971,"spinlock_wait_cycles":2304,"thread_start_us":745,"threads_started":1}
I20260811 22:23:50.678004 30109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "665aca4a5ac347b994c115b406576918" candidate_uuid: "cf71a0b394054dc2bcb028edf53b53c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7be74bc8cb54f47b3210053132a1fa6" is_pre_election: true
I20260811 22:23:50.678231 30329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "665aca4a5ac347b994c115b406576918" candidate_uuid: "cf71a0b394054dc2bcb028edf53b53c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f55973c7a3410ca1debebdf0cb707b" is_pre_election: true
I20260811 22:23:50.678678 30109 raft_consensus.cc:2454] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf71a0b394054dc2bcb028edf53b53c8 in term 0.
I20260811 22:23:50.678786 30329 raft_consensus.cc:2454] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf71a0b394054dc2bcb028edf53b53c8 in term 0.
I20260811 22:23:50.679498 29763 leader_election.cc:302] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf71a0b394054dc2bcb028edf53b53c8, d7be74bc8cb54f47b3210053132a1fa6; no voters:
I20260811 22:23:50.679978 30464 raft_consensus.cc:2781] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:23:50.680217 30464 raft_consensus.cc:482] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:23:50.680378 30464 raft_consensus.cc:3037] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:50.682968 30464 raft_consensus.cc:504] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf71a0b394054dc2bcb028edf53b53c8" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39665 } } peers { permanent_uuid: "a5f55973c7a3410ca1debebdf0cb707b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36373 } } peers { permanent_uuid: "d7be74bc8cb54f47b3210053132a1fa6" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 45987 } }
I20260811 22:23:50.683772 30464 leader_election.cc:288] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [CANDIDATE]: Term 1 election: Requested vote from peers a5f55973c7a3410ca1debebdf0cb707b (127.29.229.3:36373), d7be74bc8cb54f47b3210053132a1fa6 (127.29.229.2:45987)
I20260811 22:23:50.684314 30329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "665aca4a5ac347b994c115b406576918" candidate_uuid: "cf71a0b394054dc2bcb028edf53b53c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f55973c7a3410ca1debebdf0cb707b"
I20260811 22:23:50.684391 30109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "665aca4a5ac347b994c115b406576918" candidate_uuid: "cf71a0b394054dc2bcb028edf53b53c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7be74bc8cb54f47b3210053132a1fa6"
I20260811 22:23:50.684578 30329 raft_consensus.cc:3037] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:50.684665 30109 raft_consensus.cc:3037] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:50.687002 30329 raft_consensus.cc:2454] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf71a0b394054dc2bcb028edf53b53c8 in term 1.
I20260811 22:23:50.687019 30109 raft_consensus.cc:2454] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf71a0b394054dc2bcb028edf53b53c8 in term 1.
I20260811 22:23:50.687476 29763 leader_election.cc:302] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf71a0b394054dc2bcb028edf53b53c8, d7be74bc8cb54f47b3210053132a1fa6; no voters:
I20260811 22:23:50.687784 30464 raft_consensus.cc:2781] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:23:50.688261 30464 raft_consensus.cc:686] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [term 1 LEADER]: Becoming Leader. State: Replica: cf71a0b394054dc2bcb028edf53b53c8, State: Running, Role: LEADER
I20260811 22:23:50.688793 30464 consensus_queue.cc:237] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf71a0b394054dc2bcb028edf53b53c8" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39665 } } peers { permanent_uuid: "a5f55973c7a3410ca1debebdf0cb707b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36373 } } peers { permanent_uuid: "d7be74bc8cb54f47b3210053132a1fa6" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 45987 } }
I20260811 22:23:50.693398 29634 catalog_manager.cc:5495] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf71a0b394054dc2bcb028edf53b53c8 (127.29.229.1). New cstate: current_term: 1 leader_uuid: "cf71a0b394054dc2bcb028edf53b53c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf71a0b394054dc2bcb028edf53b53c8" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 39665 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5f55973c7a3410ca1debebdf0cb707b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7be74bc8cb54f47b3210053132a1fa6" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 45987 } health_report { overall_health: UNKNOWN } } }
W20260811 22:23:50.698287 30184 tablet.cc:2369] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:23:50.722322 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:50.716637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44158 (local address 127.29.229.1:39665)
0811 22:23:50.716815 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:50.716820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:50.716848 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:23:50.716965 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:50.716968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:50.717060 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:50.717206 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:23:50.717620 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.718345 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.719759 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.720203 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.720301 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:50.721264 (+ 963us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:23:50.721747 (+ 483us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:23:50.721905 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:50.722093 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 22:23:50.722131 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:23:50.731312 30109 raft_consensus.cc:1261] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6 [term 1 FOLLOWER]: Refusing update from remote peer cf71a0b394054dc2bcb028edf53b53c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260811 22:23:50.731272 30417 tablet.cc:2369] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:23:50.731312 30329 raft_consensus.cc:1261] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b [term 1 FOLLOWER]: Refusing update from remote peer cf71a0b394054dc2bcb028edf53b53c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:23:50.732434 30471 consensus_queue.cc:1035] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7be74bc8cb54f47b3210053132a1fa6" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 45987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:23:50.732863 30464 consensus_queue.cc:1035] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f55973c7a3410ca1debebdf0cb707b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 36373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:23:50.744127 30475 mvcc.cc:205] Tried to move back new op lower bound from 7317450877864099840 to 7317450877704835072. Current Snapshot: MvccSnapshot[applied={T|T < 7317450877864099840}]
I20260811 22:23:50.758194 29641 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:23:50.758531 29641 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:23:50.763489 29641 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 462e3922654b457fb92a43fb5feee01e: Sending DeleteTablet for 3 replicas of tablet 665aca4a5ac347b994c115b406576918
I20260811 22:23:50.764923 29844 tablet_service.cc:1514] Processing DeleteTablet for tablet 665aca4a5ac347b994c115b406576918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:50 UTC) from {username='slave'} at 127.0.0.1:44156
I20260811 22:23:50.765074 30297 tablet_service.cc:1514] Processing DeleteTablet for tablet 665aca4a5ac347b994c115b406576918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:50 UTC) from {username='slave'} at 127.0.0.1:59858
I20260811 22:23:50.766467 30077 tablet_service.cc:1514] Processing DeleteTablet for tablet 665aca4a5ac347b994c115b406576918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:50 UTC) from {username='slave'} at 127.0.0.1:51216
I20260811 22:23:50.769032 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 29740
I20260811 22:23:50.775501 30486 tablet_replica.cc:330] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8: stopping tablet replica
I20260811 22:23:50.776225 30486 raft_consensus.cc:2229] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:23:50.777123 30491 tablet_replica.cc:330] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6: stopping tablet replica
I20260811 22:23:50.777343 30486 raft_consensus.cc:2258] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:50.777568 30491 raft_consensus.cc:2229] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:23:50.778159 30491 raft_consensus.cc:2258] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:50.780469 30491 ts_tablet_manager.cc:1905] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:50.780880 30489 tablet_replica.cc:330] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b: stopping tablet replica
I20260811 22:23:50.783555 30489 raft_consensus.cc:2229] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:23:50.784082 30489 raft_consensus.cc:2258] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:50.786593 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:50.772144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44162 (local address 127.29.229.1:39665)
0811 22:23:50.772393 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:50.772402 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:50.777338 (+ 4936us) server_negotiation.cc:407] Connection header received
0811 22:23:50.777673 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:50.777681 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:50.777799 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:50.777882 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:50.778593 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.779779 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.781743 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.782410 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.782509 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:50.784743 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:50.784755 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:50.784772 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:50.785645 (+ 873us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:50.785708 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:50.785734 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:50.785818 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:50.786080 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 22:23:50.786211 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 22:23:50.780514 30486 ts_tablet_manager.cc:1905] T 665aca4a5ac347b994c115b406576918 P cf71a0b394054dc2bcb028edf53b53c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:50.787768 29942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:50.788079 30491 ts_tablet_manager.cc:1918] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:23:50.788254 30491 log.cc:1196] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/665aca4a5ac347b994c115b406576918
I20260811 22:23:50.788882 30491 ts_tablet_manager.cc:1939] T 665aca4a5ac347b994c115b406576918 P d7be74bc8cb54f47b3210053132a1fa6: Deleting consensus metadata
I20260811 22:23:50.790877 29617 catalog_manager.cc:4841] TS d7be74bc8cb54f47b3210053132a1fa6 (127.29.229.2:45987): tablet 665aca4a5ac347b994c115b406576918 (table test-table [id=2c9e4bb93eaf4136a27b462332618dd1]) successfully deleted
I20260811 22:23:50.792757 30489 ts_tablet_manager.cc:1905] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:50.798887 30489 ts_tablet_manager.cc:1918] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:23:50.799074 30489 log.cc:1196] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/665aca4a5ac347b994c115b406576918
I20260811 22:23:50.799546 30489 ts_tablet_manager.cc:1939] T 665aca4a5ac347b994c115b406576918 P a5f55973c7a3410ca1debebdf0cb707b: Deleting consensus metadata
I20260811 22:23:50.801280 29616 catalog_manager.cc:4841] TS a5f55973c7a3410ca1debebdf0cb707b (127.29.229.3:36373): tablet 665aca4a5ac347b994c115b406576918 (table test-table [id=2c9e4bb93eaf4136a27b462332618dd1]) successfully deleted
I20260811 22:23:50.893813 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 29740
W20260811 22:23:50.920408 29616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.229.1:39665 (error 108)
I20260811 22:23:50.920622 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 29957
I20260811 22:23:50.922487 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:50.921941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:39665 (local address 127.0.0.1:44170)
0811 22:23:50.922169 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:50.922307 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260811 22:23:50.922794 29616 catalog_manager.cc:4584] TS cf71a0b394054dc2bcb028edf53b53c8 (127.29.229.1:39665): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 665aca4a5ac347b994c115b406576918: Network error: Client connection negotiation failed: client connection to 127.29.229.1:39665: connect: Connection refused (error 111)
I20260811 22:23:50.933497 30445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:50.921534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51224 (local address 127.29.229.2:45987)
0811 22:23:50.921766 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:50.921772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:50.921792 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:23:50.922066 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:50.922074 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:50.922219 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:50.922353 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:50.922924 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.924045 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.926416 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:50.927369 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:50.927518 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:50.928976 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:50.928989 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:50.929010 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:50.929879 (+ 869us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:50.929974 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:50.930018 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:50.930154 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:50.932955 (+ 2801us) server_negotiation.cc:299] Negotiation successful
0811 22:23:50.933103 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:23:50.934585 30172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:50.976770 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:50.976177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:39665 (local address 127.0.0.1:44174)
0811 22:23:50.976406 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:50.976579 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 22:23:50.977877 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:50.977521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:39665 (local address 127.0.0.1:44182)
0811 22:23:50.977675 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:50.977752 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 22:23:51.027604 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 29957
I20260811 22:23:51.038983 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:51.038346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:39665 (local address 127.0.0.1:44192)
0811 22:23:51.038607 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:51.038729 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 22:23:51.040189 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:51.039720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:39665 (local address 127.0.0.1:44198)
0811 22:23:51.039921 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:51.040021 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 22:23:51.055948 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 30192
I20260811 22:23:51.063444 30448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:51.056859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59872 (local address 127.29.229.3:36373)
0811 22:23:51.057089 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:51.057098 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:51.057121 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:23:51.057357 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:51.057370 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:51.057465 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:51.057600 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:51.058022 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:51.058795 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:51.060089 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:51.060593 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:51.060732 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:51.061984 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:51.061994 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:51.062010 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:51.062629 (+ 619us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:51.062678 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:51.062698 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:51.062849 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:51.063013 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 22:23:51.063129 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 22:23:51.064225 30409 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:51.121170 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:51.120564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:39665 (local address 127.0.0.1:44204)
0811 22:23:51.120840 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:51.120962 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 22:23:51.122349 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:51.121861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:39665 (local address 127.0.0.1:44214)
0811 22:23:51.122127 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:51.122213 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260811 22:23:51.159328 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 30192
I20260811 22:23:51.185977 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 29588
I20260811 22:23:51.187556 29688 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:51.306483 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 29588
I20260811 22:23:51.306519 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:51.305614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:39665 (local address 127.0.0.1:44230)
0811 22:23:51.305872 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:51.306019 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
2026-08-11T22:23:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3121 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:23:51 dist-test-slave-jq4t krb5kdc[30509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:23:51 dist-test-slave-jq4t krb5kdc[30509](info): set up 2 sockets
Aug 11 22:23:51 dist-test-slave-jq4t krb5kdc[30509](info): commencing operation
krb5kdc: starting...
W20260811 22:23:52.737131 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.377s user 0.000s sys 0.062s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:23:52 dist-test-slave-jq4t krb5kdc[30509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:23:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:23:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:23:53.118273 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:39127
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:43503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:39127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:23:53.498188 30528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:53.498502 30528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:53.498596 30528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:53.506534 30528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:23:53.506659 30528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:53.506721 30528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:53.506774 30528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:23:53.506820 30528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:23:53.516988 30528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:43503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:39127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:39127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:53.518355 30528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:53.519821 30528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:53.532405 30537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:53.532450 30538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:53.532363 30543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:53.533577 30528 server_base.cc:1034] running on GCE node
I20260811 22:23:53.534490 30528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:53.536067 30528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:53.537350 30528 hybrid_clock.cc:648] HybridClock initialized: now 1786487033537300 us; error 58 us; skew 500 ppm
I20260811 22:23:53.537755 30528 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:23:53 dist-test-slave-jq4t krb5kdc[30509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487033, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:53.541435 30528 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:23:53.543511 30528 webserver.cc:460] Webserver started at http://127.29.229.62:39815/ using document root <none> and password file <none>
I20260811 22:23:53.544279 30528 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:53.544441 30528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:53.544797 30528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:53.547926 30528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "00beb1b2504c43b7ba402854d506b6f3"
format_stamp: "Formatted at 2026-08-11 22:23:53 on dist-test-slave-jq4t"
server_key: "59fe8057ac449362fed1484b68461e89"
server_key_iv: "f5ad91e5d852493807090d69ee426aa5"
server_key_version: "encryptionkey@0"
I20260811 22:23:53.548884 30528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "00beb1b2504c43b7ba402854d506b6f3"
format_stamp: "Formatted at 2026-08-11 22:23:53 on dist-test-slave-jq4t"
server_key: "59fe8057ac449362fed1484b68461e89"
server_key_iv: "f5ad91e5d852493807090d69ee426aa5"
server_key_version: "encryptionkey@0"
I20260811 22:23:53.556454 30528 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 22:23:53.561239 30551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:53.562620 30528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 22:23:53.562774 30528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "00beb1b2504c43b7ba402854d506b6f3"
format_stamp: "Formatted at 2026-08-11 22:23:53 on dist-test-slave-jq4t"
server_key: "59fe8057ac449362fed1484b68461e89"
server_key_iv: "f5ad91e5d852493807090d69ee426aa5"
server_key_version: "encryptionkey@0"
I20260811 22:23:53.562954 30528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:53.577595 30528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:53.582615 30528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:53.582939 30528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:53.605518 30528 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:39127
I20260811 22:23:53.605554 30657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:39127 every 8 connection(s)
I20260811 22:23:53.607034 30528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:23:53.611425 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 30528
I20260811 22:23:53.611642 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:23:53.612169 30612 external_mini_cluster.cc:1428] Setting key 73d4aa7d866eb948d4fb6261426c34a3
I20260811 22:23:53.612884 30658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:53.627076 30658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3: Bootstrap starting.
Aug 11 22:23:53 dist-test-slave-jq4t krb5kdc[30509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:23:53.630993 30658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:53.632292 30658 log.cc:824] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:53.635927 30658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3: No bootstrap required, opened a new log
I20260811 22:23:53.638631 30665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:53.614407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34934 (local address 127.29.229.62:39127)
0811 22:23:53.615237 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:53.615250 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:53.615291 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:23:53.618516 (+ 3225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:53.618543 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:53.618994 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:53.619446 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:53.621158 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:53.622491 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:53.623874 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:53.624747 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:53.624996 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:53.630089 (+ 5093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:53.630104 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:53.630121 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:53.630164 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:53.633910 (+ 3746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:53.634832 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:53.634838 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:53.634848 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:53.635037 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:53.635562 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:53.635565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:53.635567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:53.636213 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:53.636468 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:53.637017 (+ 549us) server_negotiation.cc:299] Negotiation successful
0811 22:23:53.637595 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":378,"threads_started":1}
I20260811 22:23:53.644205 30658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00beb1b2504c43b7ba402854d506b6f3" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 39127 } }
I20260811 22:23:53.644650 30658 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:53.644761 30658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00beb1b2504c43b7ba402854d506b6f3, State: Initialized, Role: FOLLOWER
I20260811 22:23:53.645373 30658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00beb1b2504c43b7ba402854d506b6f3" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 39127 } }
I20260811 22:23:53.645586 30658 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:23:53.645737 30658 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:23:53.645915 30658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:53.648687 30658 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00beb1b2504c43b7ba402854d506b6f3" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 39127 } }
I20260811 22:23:53.649322 30658 leader_election.cc:302] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00beb1b2504c43b7ba402854d506b6f3; no voters:
I20260811 22:23:53.650177 30658 leader_election.cc:288] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:23:53.650295 30668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:23:53.651113 30668 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 00beb1b2504c43b7ba402854d506b6f3, State: Running, Role: LEADER
I20260811 22:23:53.651680 30668 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00beb1b2504c43b7ba402854d506b6f3" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 39127 } }
I20260811 22:23:53.652370 30658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:23:53.655035 30670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00beb1b2504c43b7ba402854d506b6f3. Latest consensus state: current_term: 1 leader_uuid: "00beb1b2504c43b7ba402854d506b6f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00beb1b2504c43b7ba402854d506b6f3" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 39127 } } }
I20260811 22:23:53.655431 30670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:53.655169 30669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00beb1b2504c43b7ba402854d506b6f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00beb1b2504c43b7ba402854d506b6f3" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 39127 } } }
I20260811 22:23:53.656080 30669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:53.657516 30677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:23:53.663633 30677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:23:53.675562 30677 catalog_manager.cc:1347] Generated new cluster ID: bca26e94cdd844168ad13579c18d88ae
I20260811 22:23:53.675721 30677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:23:53.684139 30677 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:23:53.685774 30677 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:23:53.694109 30677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3: Generated new TSK 0
I20260811 22:23:53.695027 30677 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.229.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
Entry for principal kudu/127.29.229.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab.
I20260811 22:23:53.784816 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:39127
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:43503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:23:54.161688 30695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:54.162016 30695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:54.162287 30695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:54.171360 30695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:54.171535 30695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:54.171725 30695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:23:54.182204 30695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:43503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.1.keytab
--principal=kudu/127.29.229.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:39127
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:54.183722 30695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:54.185273 30695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:54.199434 30704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:54.199554 30711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:54.199512 30707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:54.200891 30695 server_base.cc:1034] running on GCE node
I20260811 22:23:54.201601 30695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:54.202688 30695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:54.204121 30695 hybrid_clock.cc:648] HybridClock initialized: now 1786487034203953 us; error 39 us; skew 500 ppm
I20260811 22:23:54.204564 30695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:23:54 dist-test-slave-jq4t krb5kdc[30509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487034, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:54.208084 30695 init.cc:377] Logged in from keytab as kudu/127.29.229.1@KRBTEST.COM (short username kudu)
I20260811 22:23:54.210883 30695 webserver.cc:460] Webserver started at http://127.29.229.1:45485/ using document root <none> and password file <none>
I20260811 22:23:54.212226 30695 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:54.212430 30695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:54.212875 30695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:54.216037 30695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "3f03b90e13574a96aa55f5d09f1e5735"
format_stamp: "Formatted at 2026-08-11 22:23:54 on dist-test-slave-jq4t"
server_key: "4dea3d5a080ba77aeca66bc32242fea9"
server_key_iv: "9dffe1b26ab94b8bc94cb223acd28224"
server_key_version: "encryptionkey@0"
I20260811 22:23:54.216902 30695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "3f03b90e13574a96aa55f5d09f1e5735"
format_stamp: "Formatted at 2026-08-11 22:23:54 on dist-test-slave-jq4t"
server_key: "4dea3d5a080ba77aeca66bc32242fea9"
server_key_iv: "9dffe1b26ab94b8bc94cb223acd28224"
server_key_version: "encryptionkey@0"
I20260811 22:23:54.222518 30695 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:23:54.226307 30718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:54.227687 30695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:23:54.227890 30695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "3f03b90e13574a96aa55f5d09f1e5735"
format_stamp: "Formatted at 2026-08-11 22:23:54 on dist-test-slave-jq4t"
server_key: "4dea3d5a080ba77aeca66bc32242fea9"
server_key_iv: "9dffe1b26ab94b8bc94cb223acd28224"
server_key_version: "encryptionkey@0"
I20260811 22:23:54.228091 30695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:54.246726 30695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:54.251223 30695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:54.251564 30695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:54.252677 30695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:54.254243 30695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:54.254323 30695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:54.254482 30695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:54.254549 30695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:54.292743 30695 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:40349
I20260811 22:23:54.292793 30903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:40349 every 8 connection(s)
I20260811 22:23:54.294204 30695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:23:54.294778 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 30695
I20260811 22:23:54.295044 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:23:54.295660 30612 external_mini_cluster.cc:1428] Setting key 67c0177022218d50c68c41e90868d483
Aug 11 22:23:54 dist-test-slave-jq4t krb5kdc[30509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487034, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.1@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:23:54.320073 30909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:54.297270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:47533 (local address 127.29.229.62:39127)
0811 22:23:54.297917 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:54.297924 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:54.298898 (+ 974us) server_negotiation.cc:407] Connection header received
0811 22:23:54.300923 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:54.300931 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:54.301034 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:54.301165 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:54.303532 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.304738 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.306467 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.307204 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.307360 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:54.311471 (+ 4111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:54.311480 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:54.311488 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:54.311538 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:54.315950 (+ 4412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:54.317147 (+ 1197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:54.317152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:54.317155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:54.317321 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:54.318058 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:54.318062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:54.318065 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:54.318382 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:54.318502 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:54.319599 (+ 1097us) server_negotiation.cc:299] Negotiation successful
0811 22:23:54.319804 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":281,"threads_started":1}
I20260811 22:23:54.321812 30910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:54.297835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:39127 (local address 127.29.229.1:47533)
0811 22:23:54.298716 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:54.298761 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:54.300642 (+ 1881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:54.301653 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:54.301666 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:54.302474 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:54.303311 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:54.303337 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.304983 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.304988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:54.306297 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:54.306312 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.307470 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.307476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:54.307714 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:54.308569 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:54.308596 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:54.311197 (+ 2601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:54.316150 (+ 4953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:54.316159 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:54.316173 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:54.316983 (+ 810us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:54.317538 (+ 555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:54.317547 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:54.317553 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:54.317879 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:54.318555 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:54.318570 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:54.319102 (+ 532us) client_negotiation.cc:769] Sending connection context
0811 22:23:54.319455 (+ 353us) client_negotiation.cc:240] Negotiation successful
0811 22:23:54.320095 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":368,"threads_started":1}
I20260811 22:23:54.323755 30906 heartbeater.cc:347] Connected to a master server at 127.29.229.62:39127
I20260811 22:23:54.324136 30906 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:54.325038 30906 heartbeater.cc:511] Master 127.29.229.62:39127 requested a full tablet report, sending...
I20260811 22:23:54.327630 30583 ts_manager.cc:194] Registered new tserver with Master: 3f03b90e13574a96aa55f5d09f1e5735 (127.29.229.1:40349)
I20260811 22:23:54.330641 30583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.1@KRBTEST.COM'} at 127.29.229.1:47533
WARNING: no policy specified for kudu/127.29.229.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
Entry for principal kudu/127.29.229.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab.
I20260811 22:23:54.388291 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:39127
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:43503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:23:54.769212 30914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:54.769538 30914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:54.769639 30914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:54.777305 30914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:54.777443 30914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:54.777565 30914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:23:54.787811 30914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:43503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.2.keytab
--principal=kudu/127.29.229.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:39127
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:54.789196 30914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:54.790503 30914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:54.804844 30922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:54.804883 30925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:54.804920 30923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:54.806301 30914 server_base.cc:1034] running on GCE node
I20260811 22:23:54.807202 30914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:54.808288 30914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:54.809536 30914 hybrid_clock.cc:648] HybridClock initialized: now 1786487034809466 us; error 61 us; skew 500 ppm
I20260811 22:23:54.809870 30914 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:23:54 dist-test-slave-jq4t krb5kdc[30509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487034, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:54.813263 30914 init.cc:377] Logged in from keytab as kudu/127.29.229.2@KRBTEST.COM (short username kudu)
I20260811 22:23:54.814726 30914 webserver.cc:460] Webserver started at http://127.29.229.2:43209/ using document root <none> and password file <none>
I20260811 22:23:54.815376 30914 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:54.815488 30914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:54.815758 30914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:54.818771 30914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "7372238d417e48fb82864fed6fc1930e"
format_stamp: "Formatted at 2026-08-11 22:23:54 on dist-test-slave-jq4t"
server_key: "e6eb3d6261907c4eeebf128e60055b17"
server_key_iv: "9bfedc1f8919ed60d7ec0d13cf361211"
server_key_version: "encryptionkey@0"
I20260811 22:23:54.819487 30914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "7372238d417e48fb82864fed6fc1930e"
format_stamp: "Formatted at 2026-08-11 22:23:54 on dist-test-slave-jq4t"
server_key: "e6eb3d6261907c4eeebf128e60055b17"
server_key_iv: "9bfedc1f8919ed60d7ec0d13cf361211"
server_key_version: "encryptionkey@0"
I20260811 22:23:54.824975 30914 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260811 22:23:54.828869 30938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:54.830286 30914 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:23:54.830482 30914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "7372238d417e48fb82864fed6fc1930e"
format_stamp: "Formatted at 2026-08-11 22:23:54 on dist-test-slave-jq4t"
server_key: "e6eb3d6261907c4eeebf128e60055b17"
server_key_iv: "9bfedc1f8919ed60d7ec0d13cf361211"
server_key_version: "encryptionkey@0"
I20260811 22:23:54.830715 30914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:54.851712 30914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:54.857012 30914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:54.857379 30914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:54.858960 30914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:54.860630 30914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:54.860736 30914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:54.860903 30914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:54.860981 30914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:54.899523 30914 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:35927
I20260811 22:23:54.899618 31126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:35927 every 8 connection(s)
I20260811 22:23:54.901011 30914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:23:54.910522 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 30914
I20260811 22:23:54.910732 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:23:54.911237 30612 external_mini_cluster.cc:1428] Setting key ccc117484bba5664c49538a44a2f713d
Aug 11 22:23:54 dist-test-slave-jq4t krb5kdc[30509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487034, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.2@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:23:54.923688 31132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:54.903790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:50899 (local address 127.29.229.62:39127)
0811 22:23:54.904267 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:54.904272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:54.905327 (+ 1055us) server_negotiation.cc:407] Connection header received
0811 22:23:54.907332 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:54.907338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:54.907440 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:54.907565 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:54.909373 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.910141 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.912803 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.913455 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.913615 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:54.917717 (+ 4102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:54.917722 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:54.917725 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:54.917759 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:54.920416 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:54.921256 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:54.921262 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:54.921267 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:54.921398 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:54.921865 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:54.921869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:54.921871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:54.922158 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:54.922321 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:54.923074 (+ 753us) server_negotiation.cc:299] Negotiation successful
0811 22:23:54.923305 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":229,"threads_started":1}
I20260811 22:23:54.925060 31133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:54.904354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:39127 (local address 127.29.229.2:50899)
0811 22:23:54.905152 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:54.905191 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:54.907062 (+ 1871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:54.907813 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:54.907821 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:54.908596 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:54.909179 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:54.909196 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.910316 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.910321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:54.912404 (+ 2083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:54.912433 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:54.913737 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:54.913747 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:54.914023 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:54.914774 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:54.914803 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:54.917439 (+ 2636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:54.920636 (+ 3197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:54.920645 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:54.920663 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:54.921118 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:54.921518 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:54.921523 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:54.921526 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:54.921729 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:54.922298 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:54.922307 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:54.922687 (+ 380us) client_negotiation.cc:769] Sending connection context
0811 22:23:54.922968 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 22:23:54.923550 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":346,"threads_started":1}
I20260811 22:23:54.926661 31127 heartbeater.cc:347] Connected to a master server at 127.29.229.62:39127
I20260811 22:23:54.927014 31127 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:54.927758 31127 heartbeater.cc:511] Master 127.29.229.62:39127 requested a full tablet report, sending...
I20260811 22:23:54.929382 30583 ts_manager.cc:194] Registered new tserver with Master: 7372238d417e48fb82864fed6fc1930e (127.29.229.2:35927)
I20260811 22:23:54.930212 30583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.2@KRBTEST.COM'} at 127.29.229.2:50899
WARNING: no policy specified for kudu/127.29.229.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
Entry for principal kudu/127.29.229.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab.
I20260811 22:23:54.997179 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:39127
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:43503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:23:55.334611 30906 heartbeater.cc:503] Master 127.29.229.62:39127 was elected leader, sending a full tablet report...
W20260811 22:23:55.375974 31136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:55.376300 31136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:55.376507 31136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:55.387532 31136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:55.387717 31136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:55.387893 31136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:23:55.398684 31136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:43503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.3.keytab
--principal=kudu/127.29.229.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:39127
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:55.400245 31136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:55.401684 31136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:55.417182 31145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:55.417263 31146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:55.417222 31148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:55.418602 31136 server_base.cc:1034] running on GCE node
I20260811 22:23:55.419337 31136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:55.420388 31136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:55.421664 31136 hybrid_clock.cc:648] HybridClock initialized: now 1786487035421596 us; error 55 us; skew 500 ppm
I20260811 22:23:55.422034 31136 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:23:55 dist-test-slave-jq4t krb5kdc[30509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487035, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:23:55.430255 31136 init.cc:377] Logged in from keytab as kudu/127.29.229.3@KRBTEST.COM (short username kudu)
I20260811 22:23:55.432726 31136 webserver.cc:460] Webserver started at http://127.29.229.3:44051/ using document root <none> and password file <none>
I20260811 22:23:55.433840 31136 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:55.434016 31136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:55.434443 31136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:55.439189 31136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "aa0ade38b8ad47999cb9698759bf1c0a"
format_stamp: "Formatted at 2026-08-11 22:23:55 on dist-test-slave-jq4t"
server_key: "500259f80350ad46a0acc19b106d2466"
server_key_iv: "10fea6e78c699566f9493317474c4937"
server_key_version: "encryptionkey@0"
I20260811 22:23:55.440354 31136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "aa0ade38b8ad47999cb9698759bf1c0a"
format_stamp: "Formatted at 2026-08-11 22:23:55 on dist-test-slave-jq4t"
server_key: "500259f80350ad46a0acc19b106d2466"
server_key_iv: "10fea6e78c699566f9493317474c4937"
server_key_version: "encryptionkey@0"
I20260811 22:23:55.448210 31136 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260811 22:23:55.452972 31157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:55.454587 31136 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:23:55.454797 31136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "aa0ade38b8ad47999cb9698759bf1c0a"
format_stamp: "Formatted at 2026-08-11 22:23:55 on dist-test-slave-jq4t"
server_key: "500259f80350ad46a0acc19b106d2466"
server_key_iv: "10fea6e78c699566f9493317474c4937"
server_key_version: "encryptionkey@0"
I20260811 22:23:55.455039 31136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:55.505551 31136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:55.512923 31136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:55.513305 31136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:55.514515 31136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:55.516456 31136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:55.516750 31136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 22:23:55.516973 31136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:55.517056 31136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:55.559614 31136 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:35351
I20260811 22:23:55.559648 31333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:35351 every 8 connection(s)
I20260811 22:23:55.561278 31136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:23:55.569139 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 31136
I20260811 22:23:55.569360 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:23:55.569918 30612 external_mini_cluster.cc:1428] Setting key 7a2873d2297a876c8a86ebb13a470e4c
Aug 11 22:23:55 dist-test-slave-jq4t krb5kdc[30509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487035, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.3@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:23:55.591367 31338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.565038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:51687 (local address 127.29.229.62:39127)
0811 22:23:55.566385 (+ 1347us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:55.566392 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:55.569569 (+ 3177us) server_negotiation.cc:407] Connection header received
0811 22:23:55.572691 (+ 3122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:55.572700 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:55.572814 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:55.572961 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:55.574912 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.576524 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.578182 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.579025 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.579181 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.583380 (+ 4199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:55.583385 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:55.583388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:55.583421 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:55.587518 (+ 4097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:55.588618 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:55.588622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:55.588624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:55.588752 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:55.589321 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:55.589324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:55.589326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:55.589619 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:55.589744 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:55.590876 (+ 1132us) server_negotiation.cc:299] Negotiation successful
0811 22:23:55.591080 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1100,"thread_start_us":2658,"threads_started":1}
I20260811 22:23:55.593146 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.565762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:39127 (local address 127.29.229.3:51687)
0811 22:23:55.569353 (+ 3591us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:55.569413 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:55.571902 (+ 2489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:55.573269 (+ 1367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:55.573283 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:55.574088 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:55.574709 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.574728 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.576688 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.576695 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.578016 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.578029 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.579308 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.579314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.579534 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.580379 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:23:55.580409 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:55.583130 (+ 2721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:55.587673 (+ 4543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:55.587686 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:55.587709 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:55.588461 (+ 752us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:55.588891 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:23:55.588898 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:23:55.588905 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:23:55.589206 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:23:55.589771 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:55.589783 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:55.590345 (+ 562us) client_negotiation.cc:769] Sending connection context
0811 22:23:55.590710 (+ 365us) client_negotiation.cc:240] Negotiation successful
0811 22:23:55.591405 (+ 695us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3230,"thread_start_us":2866,"threads_started":1}
I20260811 22:23:55.594799 31335 heartbeater.cc:347] Connected to a master server at 127.29.229.62:39127
I20260811 22:23:55.595151 31335 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:55.595921 31335 heartbeater.cc:511] Master 127.29.229.62:39127 requested a full tablet report, sending...
I20260811 22:23:55.598160 30583 ts_manager.cc:194] Registered new tserver with Master: aa0ade38b8ad47999cb9698759bf1c0a (127.29.229.3:35351)
I20260811 22:23:55.599357 30583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.229.3@KRBTEST.COM'} at 127.29.229.3:51687
I20260811 22:23:55.600322 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:23:55.623010 31338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.606947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33888 (local address 127.29.229.62:39127)
0811 22:23:55.607159 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:55.607166 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:55.607627 (+ 461us) server_negotiation.cc:407] Connection header received
0811 22:23:55.607754 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:55.607759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:55.607868 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:55.607985 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:55.609718 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.610905 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.612173 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.612904 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.613026 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.616129 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:55.616135 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:55.616139 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:55.616175 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:55.619706 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:55.621012 (+ 1306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:55.621017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:55.621020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:55.621169 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:55.621547 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:55.621550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:55.621552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:55.621915 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:55.622119 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:55.622540 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 22:23:55.622754 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:23:55.628463 30581 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33888:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:23:55.637655 30581 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:23:55.668685 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.652956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:35351 (local address 127.0.0.1:33686)
0811 22:23:55.655182 (+ 2226us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:55.655225 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:55.655452 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:55.656550 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:55.656561 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:55.656605 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:55.657289 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.657308 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.660679 (+ 3371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.660688 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.664548 (+ 3860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.664572 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.667561 (+ 2989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.667570 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.667833 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.667904 (+ 71us) client_negotiation.cc:769] Sending connection context
0811 22:23:55.668027 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 22:23:55.668243 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1966,"thread_start_us":1765,"threads_started":1}
I20260811 22:23:55.669391 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.653344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33686 (local address 127.29.229.3:35351)
0811 22:23:55.654366 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:55.654378 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:55.655705 (+ 1327us) server_negotiation.cc:407] Connection header received
0811 22:23:55.655850 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:55.655861 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:55.656177 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:55.657621 (+ 1444us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:55.658039 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.660441 (+ 2402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.664831 (+ 4390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.667251 (+ 2420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.668498 (+ 1247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.668753 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:55.668861 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 22:23:55.669031 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":839,"thread_start_us":690,"threads_started":1}
I20260811 22:23:55.671528 31360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.653224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:40349 (local address 127.0.0.1:37472)
0811 22:23:55.657833 (+ 4609us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:55.657852 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:55.657989 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:55.659576 (+ 1587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:55.659585 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:55.659622 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:55.660202 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.660219 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.664866 (+ 4647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.664875 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.669059 (+ 4184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.669078 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.670729 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.670737 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.670935 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.670969 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 22:23:55.671064 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 22:23:55.671193 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4486,"thread_start_us":4331,"threads_started":1}
I20260811 22:23:55.671627 31364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.654070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37472 (local address 127.29.229.1:40349)
0811 22:23:55.655606 (+ 1536us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:55.655618 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:55.658816 (+ 3198us) server_negotiation.cc:407] Connection header received
0811 22:23:55.658957 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:55.658968 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:55.659278 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:55.660507 (+ 1229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:55.660814 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.664657 (+ 3843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.669207 (+ 4550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.670471 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.670661 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.670937 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:55.671127 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 22:23:55.671298 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":371,"threads_started":1}
I20260811 22:23:55.673997 31230 tablet_service.cc:1467] Processing CreateTablet for tablet 6f7dc0048c3143178eb9b136dbf060c9 (DEFAULT_TABLE table=test-table [id=d344fb6b93d748edadf59cd5acb1e0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:55.675541 31230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f7dc0048c3143178eb9b136dbf060c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:55.675320 30796 tablet_service.cc:1467] Processing CreateTablet for tablet 6f7dc0048c3143178eb9b136dbf060c9 (DEFAULT_TABLE table=test-table [id=d344fb6b93d748edadf59cd5acb1e0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:55.676550 31367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.655952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:35927 (local address 127.0.0.1:42626)
0811 22:23:55.667929 (+ 11977us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:55.667951 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:55.668078 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:55.669271 (+ 1193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:55.669277 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:55.669310 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:55.669676 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.669689 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.671654 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.671661 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.673615 (+ 1954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.673632 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.675653 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.675660 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.675857 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.675892 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 22:23:55.676004 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 22:23:55.676142 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11809,"thread_start_us":11592,"threads_started":1}
I20260811 22:23:55.676772 30796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f7dc0048c3143178eb9b136dbf060c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:55.676805 31366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.654633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42626 (local address 127.29.229.2:35927)
0811 22:23:55.668420 (+ 13787us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:55.668428 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:55.668460 (+ 32us) server_negotiation.cc:407] Connection header received
0811 22:23:55.668584 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:55.668592 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:55.668914 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:55.669664 (+ 750us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:55.669916 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.671466 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.673818 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.675309 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.675469 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.675671 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:55.676119 (+ 448us) server_negotiation.cc:299] Negotiation successful
0811 22:23:55.676304 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13550,"spinlock_wait_cycles":21888,"thread_start_us":13517,"threads_started":1}
I20260811 22:23:55.680876 31021 tablet_service.cc:1467] Processing CreateTablet for tablet 6f7dc0048c3143178eb9b136dbf060c9 (DEFAULT_TABLE table=test-table [id=d344fb6b93d748edadf59cd5acb1e0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:55.682410 31021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f7dc0048c3143178eb9b136dbf060c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:55.689061 31368 tablet_bootstrap.cc:492] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a: Bootstrap starting.
I20260811 22:23:55.692701 31369 tablet_bootstrap.cc:492] T 6f7dc0048c3143178eb9b136dbf060c9 P 3f03b90e13574a96aa55f5d09f1e5735: Bootstrap starting.
I20260811 22:23:55.696249 31368 tablet_bootstrap.cc:654] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:55.697700 31369 tablet_bootstrap.cc:654] T 6f7dc0048c3143178eb9b136dbf060c9 P 3f03b90e13574a96aa55f5d09f1e5735: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:55.697875 31368 log.cc:824] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:55.698709 31370 tablet_bootstrap.cc:492] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e: Bootstrap starting.
I20260811 22:23:55.699126 31369 log.cc:824] T 6f7dc0048c3143178eb9b136dbf060c9 P 3f03b90e13574a96aa55f5d09f1e5735: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:55.702291 31368 tablet_bootstrap.cc:492] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a: No bootstrap required, opened a new log
I20260811 22:23:55.702605 31368 ts_tablet_manager.cc:1397] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.002s
I20260811 22:23:55.702607 31369 tablet_bootstrap.cc:492] T 6f7dc0048c3143178eb9b136dbf060c9 P 3f03b90e13574a96aa55f5d09f1e5735: No bootstrap required, opened a new log
I20260811 22:23:55.702709 31370 tablet_bootstrap.cc:654] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:55.702924 31369 ts_tablet_manager.cc:1397] T 6f7dc0048c3143178eb9b136dbf060c9 P 3f03b90e13574a96aa55f5d09f1e5735: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 22:23:55.704145 31370 log.cc:824] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:55.708570 31370 tablet_bootstrap.cc:492] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e: No bootstrap required, opened a new log
I20260811 22:23:55.708882 31370 ts_tablet_manager.cc:1397] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.003s
I20260811 22:23:55.711670 31368 raft_consensus.cc:348] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa0ade38b8ad47999cb9698759bf1c0a" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35351 } } peers { permanent_uuid: "3f03b90e13574a96aa55f5d09f1e5735" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 40349 } } peers { permanent_uuid: "7372238d417e48fb82864fed6fc1930e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35927 } }
I20260811 22:23:55.712096 31368 raft_consensus.cc:374] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:55.712201 31368 raft_consensus.cc:729] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa0ade38b8ad47999cb9698759bf1c0a, State: Initialized, Role: FOLLOWER
I20260811 22:23:55.712921 31368 consensus_queue.cc:260] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa0ade38b8ad47999cb9698759bf1c0a" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35351 } } peers { permanent_uuid: "3f03b90e13574a96aa55f5d09f1e5735" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 40349 } } peers { permanent_uuid: "7372238d417e48fb82864fed6fc1930e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35927 } }
I20260811 22:23:55.714253 31335 heartbeater.cc:503] Master 127.29.229.62:39127 was elected leader, sending a full tablet report...
I20260811 22:23:55.715231 31369 raft_consensus.cc:348] T 6f7dc0048c3143178eb9b136dbf060c9 P 3f03b90e13574a96aa55f5d09f1e5735 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa0ade38b8ad47999cb9698759bf1c0a" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35351 } } peers { permanent_uuid: "3f03b90e13574a96aa55f5d09f1e5735" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 40349 } } peers { permanent_uuid: "7372238d417e48fb82864fed6fc1930e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35927 } }
I20260811 22:23:55.715698 31369 raft_consensus.cc:374] T 6f7dc0048c3143178eb9b136dbf060c9 P 3f03b90e13574a96aa55f5d09f1e5735 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:55.715790 31369 raft_consensus.cc:729] T 6f7dc0048c3143178eb9b136dbf060c9 P 3f03b90e13574a96aa55f5d09f1e5735 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f03b90e13574a96aa55f5d09f1e5735, State: Initialized, Role: FOLLOWER
I20260811 22:23:55.716355 31368 ts_tablet_manager.cc:1428] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
I20260811 22:23:55.716432 31369 consensus_queue.cc:260] T 6f7dc0048c3143178eb9b136dbf060c9 P 3f03b90e13574a96aa55f5d09f1e5735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa0ade38b8ad47999cb9698759bf1c0a" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35351 } } peers { permanent_uuid: "3f03b90e13574a96aa55f5d09f1e5735" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 40349 } } peers { permanent_uuid: "7372238d417e48fb82864fed6fc1930e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35927 } }
I20260811 22:23:55.718192 31369 ts_tablet_manager.cc:1428] T 6f7dc0048c3143178eb9b136dbf060c9 P 3f03b90e13574a96aa55f5d09f1e5735: Time spent starting tablet: real 0.015s user 0.012s sys 0.005s
I20260811 22:23:55.720346 31370 raft_consensus.cc:348] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa0ade38b8ad47999cb9698759bf1c0a" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35351 } } peers { permanent_uuid: "3f03b90e13574a96aa55f5d09f1e5735" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 40349 } } peers { permanent_uuid: "7372238d417e48fb82864fed6fc1930e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35927 } }
I20260811 22:23:55.720885 31370 raft_consensus.cc:374] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:55.721016 31370 raft_consensus.cc:729] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7372238d417e48fb82864fed6fc1930e, State: Initialized, Role: FOLLOWER
I20260811 22:23:55.721747 31370 consensus_queue.cc:260] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa0ade38b8ad47999cb9698759bf1c0a" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35351 } } peers { permanent_uuid: "3f03b90e13574a96aa55f5d09f1e5735" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 40349 } } peers { permanent_uuid: "7372238d417e48fb82864fed6fc1930e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35927 } }
I20260811 22:23:55.723089 31127 heartbeater.cc:503] Master 127.29.229.62:39127 was elected leader, sending a full tablet report...
I20260811 22:23:55.723680 31370 ts_tablet_manager.cc:1428] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
W20260811 22:23:55.797448 30907 tablet.cc:2369] T 6f7dc0048c3143178eb9b136dbf060c9 P 3f03b90e13574a96aa55f5d09f1e5735: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:23:55.813684 31336 tablet.cc:2369] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:23:55.903599 31128 tablet.cc:2369] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:23:55.910710 31375 raft_consensus.cc:482] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:23:55.911042 31375 raft_consensus.cc:504] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa0ade38b8ad47999cb9698759bf1c0a" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35351 } } peers { permanent_uuid: "3f03b90e13574a96aa55f5d09f1e5735" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 40349 } } peers { permanent_uuid: "7372238d417e48fb82864fed6fc1930e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35927 } }
I20260811 22:23:55.913223 31375 leader_election.cc:288] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f03b90e13574a96aa55f5d09f1e5735 (127.29.229.1:40349), 7372238d417e48fb82864fed6fc1930e (127.29.229.2:35927)
I20260811 22:23:55.919260 31364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.913743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:49971 (local address 127.29.229.1:40349)
0811 22:23:55.913961 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:55.913967 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:55.914216 (+ 249us) server_negotiation.cc:407] Connection header received
0811 22:23:55.914452 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:55.914459 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:55.914527 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:55.914628 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:55.915027 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.915819 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.917555 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.918349 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.918535 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.918611 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:55.918893 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 22:23:55.919017 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:23:55.919299 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.913731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:40349 (local address 127.29.229.3:49971)
0811 22:23:55.914108 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:55.914124 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:55.914297 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:55.914657 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:55.914663 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:55.914692 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:55.914886 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.914895 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.915977 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.915989 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.917394 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.917407 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.918523 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.918529 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.918725 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.918749 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:23:55.918863 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 22:23:55.919000 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":19}
I20260811 22:23:55.919895 31366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.913762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:32883 (local address 127.29.229.2:35927)
0811 22:23:55.914008 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:55.914017 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:55.914780 (+ 763us) server_negotiation.cc:407] Connection header received
0811 22:23:55.914995 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:55.915004 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:55.915059 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:55.915159 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:55.915714 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.916402 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.918147 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.918933 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.919109 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.919181 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:55.919533 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 22:23:55.919659 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 22:23:55.920663 31381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.913774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:35927 (local address 127.29.229.3:32883)
0811 22:23:55.914660 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:55.914674 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:55.914848 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:55.915176 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:55.915180 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:55.915205 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:55.915542 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.915550 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.916584 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.916590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.917975 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:55.917987 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.919142 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.919148 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:55.919309 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.919360 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 22:23:55.920107 (+ 747us) client_negotiation.cc:240] Negotiation successful
0811 22:23:55.920243 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"mutex_wait_us":21,"thread_start_us":542,"threads_started":1}
I20260811 22:23:55.920511 30824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f7dc0048c3143178eb9b136dbf060c9" candidate_uuid: "aa0ade38b8ad47999cb9698759bf1c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f03b90e13574a96aa55f5d09f1e5735" is_pre_election: true
I20260811 22:23:55.921108 30824 raft_consensus.cc:2454] T 6f7dc0048c3143178eb9b136dbf060c9 P 3f03b90e13574a96aa55f5d09f1e5735 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa0ade38b8ad47999cb9698759bf1c0a in term 0.
I20260811 22:23:55.921746 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f7dc0048c3143178eb9b136dbf060c9" candidate_uuid: "aa0ade38b8ad47999cb9698759bf1c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7372238d417e48fb82864fed6fc1930e" is_pre_election: true
I20260811 22:23:55.921870 31162 leader_election.cc:302] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f03b90e13574a96aa55f5d09f1e5735, aa0ade38b8ad47999cb9698759bf1c0a; no voters:
I20260811 22:23:55.922237 31051 raft_consensus.cc:2454] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa0ade38b8ad47999cb9698759bf1c0a in term 0.
I20260811 22:23:55.922276 31375 raft_consensus.cc:2781] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:23:55.922451 31375 raft_consensus.cc:482] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:23:55.922576 31375 raft_consensus.cc:3037] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:55.925113 31375 raft_consensus.cc:504] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa0ade38b8ad47999cb9698759bf1c0a" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35351 } } peers { permanent_uuid: "3f03b90e13574a96aa55f5d09f1e5735" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 40349 } } peers { permanent_uuid: "7372238d417e48fb82864fed6fc1930e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35927 } }
I20260811 22:23:55.925979 31375 leader_election.cc:288] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [CANDIDATE]: Term 1 election: Requested vote from peers 3f03b90e13574a96aa55f5d09f1e5735 (127.29.229.1:40349), 7372238d417e48fb82864fed6fc1930e (127.29.229.2:35927)
I20260811 22:23:55.926388 30824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f7dc0048c3143178eb9b136dbf060c9" candidate_uuid: "aa0ade38b8ad47999cb9698759bf1c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f03b90e13574a96aa55f5d09f1e5735"
I20260811 22:23:55.926518 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f7dc0048c3143178eb9b136dbf060c9" candidate_uuid: "aa0ade38b8ad47999cb9698759bf1c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7372238d417e48fb82864fed6fc1930e"
I20260811 22:23:55.926614 30824 raft_consensus.cc:3037] T 6f7dc0048c3143178eb9b136dbf060c9 P 3f03b90e13574a96aa55f5d09f1e5735 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:55.926776 31051 raft_consensus.cc:3037] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:55.929036 30824 raft_consensus.cc:2454] T 6f7dc0048c3143178eb9b136dbf060c9 P 3f03b90e13574a96aa55f5d09f1e5735 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa0ade38b8ad47999cb9698759bf1c0a in term 1.
I20260811 22:23:55.929201 31051 raft_consensus.cc:2454] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa0ade38b8ad47999cb9698759bf1c0a in term 1.
I20260811 22:23:55.929518 31162 leader_election.cc:302] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f03b90e13574a96aa55f5d09f1e5735, aa0ade38b8ad47999cb9698759bf1c0a; no voters:
I20260811 22:23:55.929821 31375 raft_consensus.cc:2781] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:23:55.930362 31375 raft_consensus.cc:686] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [term 1 LEADER]: Becoming Leader. State: Replica: aa0ade38b8ad47999cb9698759bf1c0a, State: Running, Role: LEADER
I20260811 22:23:55.930886 31375 consensus_queue.cc:237] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa0ade38b8ad47999cb9698759bf1c0a" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35351 } } peers { permanent_uuid: "3f03b90e13574a96aa55f5d09f1e5735" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 40349 } } peers { permanent_uuid: "7372238d417e48fb82864fed6fc1930e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35927 } }
I20260811 22:23:55.936201 30581 catalog_manager.cc:5495] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a reported cstate change: term changed from 0 to 1, leader changed from <none> to aa0ade38b8ad47999cb9698759bf1c0a (127.29.229.3). New cstate: current_term: 1 leader_uuid: "aa0ade38b8ad47999cb9698759bf1c0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa0ade38b8ad47999cb9698759bf1c0a" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 35351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f03b90e13574a96aa55f5d09f1e5735" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 40349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7372238d417e48fb82864fed6fc1930e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35927 } health_report { overall_health: UNKNOWN } } }
I20260811 22:23:55.966603 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:55.960861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33696 (local address 127.29.229.3:35351)
0811 22:23:55.961112 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:55.961119 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:55.961230 (+ 111us) server_negotiation.cc:407] Connection header received
0811 22:23:55.961490 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:55.961494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:55.961644 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:55.961807 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:23:55.962192 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.962947 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.964355 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:55.964834 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:55.964971 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:55.965472 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:23:55.965978 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:23:55.966174 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:55.966233 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 22:23:55.966352 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:23:55.975356 30824 raft_consensus.cc:1261] T 6f7dc0048c3143178eb9b136dbf060c9 P 3f03b90e13574a96aa55f5d09f1e5735 [term 1 FOLLOWER]: Refusing update from remote peer aa0ade38b8ad47999cb9698759bf1c0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:23:55.975355 31051 raft_consensus.cc:1261] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e [term 1 FOLLOWER]: Refusing update from remote peer aa0ade38b8ad47999cb9698759bf1c0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:23:55.976416 31383 consensus_queue.cc:1035] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7372238d417e48fb82864fed6fc1930e" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 35927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:23:55.976859 31375 consensus_queue.cc:1035] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f03b90e13574a96aa55f5d09f1e5735" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 40349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:23:55.989984 31388 mvcc.cc:205] Tried to move back new op lower bound from 7317450899343126528 to 7317450899177164800. Current Snapshot: MvccSnapshot[applied={T|T < 7317450899343126528}]
I20260811 22:23:56.006855 30581 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:23:56.007153 30581 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:23:56.011374 30581 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00beb1b2504c43b7ba402854d506b6f3: Sending DeleteTablet for 3 replicas of tablet 6f7dc0048c3143178eb9b136dbf060c9
I20260811 22:23:56.013307 30796 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f7dc0048c3143178eb9b136dbf060c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:56 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:37472
I20260811 22:23:56.013403 31230 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f7dc0048c3143178eb9b136dbf060c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:56 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:33686
I20260811 22:23:56.014147 31021 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f7dc0048c3143178eb9b136dbf060c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:56 UTC) from {username='kudu', principal='kudu/127.29.229.62@KRBTEST.COM'} at 127.0.0.1:42626
I20260811 22:23:56.017061 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 30695
I20260811 22:23:56.037930 31410 tablet_replica.cc:330] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e: stopping tablet replica
I20260811 22:23:56.038669 31410 raft_consensus.cc:2229] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:23:56.039445 31410 raft_consensus.cc:2258] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:56.040151 31409 tablet_replica.cc:330] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a: stopping tablet replica
I20260811 22:23:56.042356 31410 ts_tablet_manager.cc:1905] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 22:23:56 dist-test-slave-jq4t krb5kdc[30509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.1@KRBTEST.COM
I20260811 22:23:56.044345 31409 raft_consensus.cc:2229] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:23:56.046542 31409 raft_consensus.cc:2258] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:56.046692 31404 tablet_replica.cc:330] T 6f7dc0048c3143178eb9b136dbf060c9 P 3f03b90e13574a96aa55f5d09f1e5735: stopping tablet replica
I20260811 22:23:56.047219 31404 raft_consensus.cc:2229] T 6f7dc0048c3143178eb9b136dbf060c9 P 3f03b90e13574a96aa55f5d09f1e5735 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:23:56.047775 31404 raft_consensus.cc:2258] T 6f7dc0048c3143178eb9b136dbf060c9 P 3f03b90e13574a96aa55f5d09f1e5735 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:56.051028 31409 ts_tablet_manager.cc:1905] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:56.051715 31404 ts_tablet_manager.cc:1905] T 6f7dc0048c3143178eb9b136dbf060c9 P 3f03b90e13574a96aa55f5d09f1e5735: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:56.053594 31410 ts_tablet_manager.cc:1918] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:23:56.056406 31364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:56.018679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37486 (local address 127.29.229.1:40349)
0811 22:23:56.018907 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:56.018915 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:56.030198 (+ 11283us) server_negotiation.cc:407] Connection header received
0811 22:23:56.030876 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:56.030887 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:56.031018 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:56.031113 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:56.033635 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:56.035136 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:56.037843 (+ 2707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:56.038489 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:56.038607 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:56.047289 (+ 8682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:56.047305 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:56.047324 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:56.047398 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:56.052131 (+ 4733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:56.053319 (+ 1188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:56.053341 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:56.053345 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:56.053544 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:56.054196 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:56.054201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:56.054204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:56.054729 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:56.055047 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:56.055672 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 22:23:56.055949 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:23:56.056813 31410 log.cc:1196] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/6f7dc0048c3143178eb9b136dbf060c9
I20260811 22:23:56.057999 31410 ts_tablet_manager.cc:1939] T 6f7dc0048c3143178eb9b136dbf060c9 P 7372238d417e48fb82864fed6fc1930e: Deleting consensus metadata
I20260811 22:23:56.058130 30897 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:56.062974 30554 catalog_manager.cc:4841] TS 7372238d417e48fb82864fed6fc1930e (127.29.229.2:35927): tablet 6f7dc0048c3143178eb9b136dbf060c9 (table test-table [id=d344fb6b93d748edadf59cd5acb1e0fa]) successfully deleted
I20260811 22:23:56.065718 31409 ts_tablet_manager.cc:1918] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:23:56.065933 31409 log.cc:1196] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/6f7dc0048c3143178eb9b136dbf060c9
I20260811 22:23:56.066565 31409 ts_tablet_manager.cc:1939] T 6f7dc0048c3143178eb9b136dbf060c9 P aa0ade38b8ad47999cb9698759bf1c0a: Deleting consensus metadata
I20260811 22:23:56.069478 30552 catalog_manager.cc:4841] TS aa0ade38b8ad47999cb9698759bf1c0a (127.29.229.3:35351): tablet 6f7dc0048c3143178eb9b136dbf060c9 (table test-table [id=d344fb6b93d748edadf59cd5acb1e0fa]) successfully deleted
I20260811 22:23:56.180688 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 30695
W20260811 22:23:56.209679 30555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.229.1:40349 (error 108)
I20260811 22:23:56.210559 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 30914
I20260811 22:23:56.212697 31416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:56.211626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:40349 (local address 127.0.0.1:37492)
0811 22:23:56.212292 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:56.212435 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:40349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":432,"threads_started":1}
W20260811 22:23:56.213003 30555 catalog_manager.cc:4584] TS 3f03b90e13574a96aa55f5d09f1e5735 (127.29.229.1:40349): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f7dc0048c3143178eb9b136dbf060c9: Network error: Client connection negotiation failed: client connection to 127.29.229.1:40349: connect: Connection refused (error 111)
Aug 11 22:23:56 dist-test-slave-jq4t krb5kdc[30509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.2@KRBTEST.COM
I20260811 22:23:56.230265 31366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:56.211664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42638 (local address 127.29.229.2:35927)
0811 22:23:56.211900 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:56.211910 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:56.212263 (+ 353us) server_negotiation.cc:407] Connection header received
0811 22:23:56.212379 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:56.212384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:56.212520 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:56.212633 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:56.214944 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:56.215715 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:56.217750 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:56.218169 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:56.218280 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:56.223814 (+ 5534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:56.223834 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:56.223850 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:56.223894 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:56.227134 (+ 3240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:56.227959 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:56.227973 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:56.227976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:56.228113 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:56.228526 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:56.228533 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:56.228536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:56.228952 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:56.229294 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:56.229647 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 22:23:56.229883 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:23:56.230973 31120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:56.266855 31416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:56.266376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:40349 (local address 127.0.0.1:37504)
0811 22:23:56.266593 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:56.266705 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:40349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 22:23:56.268561 31416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:56.268100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:40349 (local address 127.0.0.1:37516)
0811 22:23:56.268257 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:56.268361 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:40349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 22:23:56.329066 31416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:56.328621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:40349 (local address 127.0.0.1:37530)
0811 22:23:56.328825 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:56.328914 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:40349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 22:23:56.330156 31416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:56.329813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:40349 (local address 127.0.0.1:37536)
0811 22:23:56.329957 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:56.330031 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:40349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 22:23:56.372713 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 30914
I20260811 22:23:56.399470 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 31136
I20260811 22:23:56.411453 31416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:56.410508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:40349 (local address 127.0.0.1:37542)
0811 22:23:56.410971 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:56.411152 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:40349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
Aug 11 22:23:56 dist-test-slave-jq4t krb5kdc[30509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.3@KRBTEST.COM
I20260811 22:23:56.413854 31416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:56.412464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:40349 (local address 127.0.0.1:37548)
0811 22:23:56.413470 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:56.413591 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:40349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":763}
I20260811 22:23:56.423336 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:56.400411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33698 (local address 127.29.229.3:35351)
0811 22:23:56.400633 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:56.400640 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:56.400664 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:23:56.400928 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:56.400934 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:56.401028 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:56.401189 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:56.402509 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:56.404176 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:56.406719 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:56.407595 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:56.407802 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:56.414779 (+ 6977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:56.414802 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:56.414825 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:23:56.414901 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:56.419580 (+ 4679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:56.420674 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:56.420684 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:56.420688 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:56.420875 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:23:56.421386 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:23:56.421397 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:23:56.421402 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:23:56.421976 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:56.422344 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:56.422653 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 22:23:56.422923 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 22:23:56.424209 31327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:56.555042 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 31136
I20260811 22:23:56.587185 31416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:56.586495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:40349 (local address 127.0.0.1:37552)
0811 22:23:56.586786 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:56.586940 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:40349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260811 22:23:56.587476 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 30528
I20260811 22:23:56.588466 31416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:56.588054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:40349 (local address 127.0.0.1:37554)
0811 22:23:56.588247 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:56.588316 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:40349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 22:23:56.588685 30649 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:56.702463 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 30528
2026-08-11T22:23:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5399 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T22:23:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:23:56Z Disabled control of system clock
I20260811 22:23:56.819484 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:43581
--webserver_interface=127.29.229.62
--webserver_port=0
--builtin_ntp_servers=127.29.229.20:41775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:43581
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:23:57.206845 31426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:57.207288 31426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:57.207463 31426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:57.221031 31426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:23:57.221199 31426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:57.221266 31426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:57.221356 31426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:23:57.221444 31426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:23:57.238435 31426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:41775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:43581
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:43581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:57.240293 31426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:57.243184 31426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:57.263371 31436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:57.264113 31445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:57.264189 31442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:57.265293 31426 server_base.cc:1034] running on GCE node
I20260811 22:23:57.266497 31426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:57.268926 31426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:57.270184 31426 hybrid_clock.cc:648] HybridClock initialized: now 1786487037270121 us; error 60 us; skew 500 ppm
I20260811 22:23:57.270635 31426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:23:57.273535 31426 webserver.cc:460] Webserver started at http://127.29.229.62:39303/ using document root <none> and password file <none>
I20260811 22:23:57.274291 31426 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:57.274386 31426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:57.274796 31426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:57.278044 31426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "1a117c717163475ba318823631ffec9b"
format_stamp: "Formatted at 2026-08-11 22:23:57 on dist-test-slave-jq4t"
server_key: "a8229f63b916c38380951f6e437458d2"
server_key_iv: "6b99b1a93eb6e97da00935c4ea65efbe"
server_key_version: "encryptionkey@0"
I20260811 22:23:57.278906 31426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "1a117c717163475ba318823631ffec9b"
format_stamp: "Formatted at 2026-08-11 22:23:57 on dist-test-slave-jq4t"
server_key: "a8229f63b916c38380951f6e437458d2"
server_key_iv: "6b99b1a93eb6e97da00935c4ea65efbe"
server_key_version: "encryptionkey@0"
I20260811 22:23:57.284904 31426 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 22:23:57.288633 31461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:57.290038 31426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:23:57.290238 31426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "1a117c717163475ba318823631ffec9b"
format_stamp: "Formatted at 2026-08-11 22:23:57 on dist-test-slave-jq4t"
server_key: "a8229f63b916c38380951f6e437458d2"
server_key_iv: "6b99b1a93eb6e97da00935c4ea65efbe"
server_key_version: "encryptionkey@0"
I20260811 22:23:57.290482 31426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:57.303432 31426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:57.304414 31426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:57.304813 31426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:57.325976 31426 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:43581
I20260811 22:23:57.326014 31546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:43581 every 8 connection(s)
I20260811 22:23:57.327440 31426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:23:57.331489 31547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:57.334955 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 31426
I20260811 22:23:57.335144 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:23:57.335729 30612 external_mini_cluster.cc:1428] Setting key 8208b549933ce9a9aabf3544695e72f8
I20260811 22:23:57.344894 31547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b: Bootstrap starting.
I20260811 22:23:57.348634 31547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:57.349735 31547 log.cc:824] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:57.349855 31552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:57.337840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58622 (local address 127.29.229.62:43581)
0811 22:23:57.338843 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:57.338853 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:57.338895 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:23:57.340600 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:57.340626 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:57.341072 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:57.341394 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:57.342022 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:57.343351 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:57.345244 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:57.345938 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:57.346101 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:57.347450 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:57.347457 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:57.347470 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:57.348013 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:57.348105 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:57.348127 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:57.348224 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:57.348534 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 22:23:57.348874 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":396,"threads_started":1}
I20260811 22:23:57.353124 31547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b: No bootstrap required, opened a new log
I20260811 22:23:57.361001 31547 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a117c717163475ba318823631ffec9b" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43581 } }
I20260811 22:23:57.361434 31547 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:57.361554 31547 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a117c717163475ba318823631ffec9b, State: Initialized, Role: FOLLOWER
I20260811 22:23:57.362329 31547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a117c717163475ba318823631ffec9b" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43581 } }
I20260811 22:23:57.362651 31547 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:23:57.362818 31547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:23:57.363008 31547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:57.365899 31547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a117c717163475ba318823631ffec9b" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43581 } }
I20260811 22:23:57.366605 31547 leader_election.cc:302] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a117c717163475ba318823631ffec9b; no voters:
I20260811 22:23:57.367398 31547 leader_election.cc:288] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:23:57.367569 31556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:23:57.368316 31556 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b [term 1 LEADER]: Becoming Leader. State: Replica: 1a117c717163475ba318823631ffec9b, State: Running, Role: LEADER
I20260811 22:23:57.368891 31556 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a117c717163475ba318823631ffec9b" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43581 } }
I20260811 22:23:57.369689 31547 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:23:57.372251 31560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a117c717163475ba318823631ffec9b. Latest consensus state: current_term: 1 leader_uuid: "1a117c717163475ba318823631ffec9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a117c717163475ba318823631ffec9b" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43581 } } }
I20260811 22:23:57.373174 31560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:57.373821 31557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a117c717163475ba318823631ffec9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a117c717163475ba318823631ffec9b" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 43581 } } }
I20260811 22:23:57.374522 31557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b [sys.catalog]: This master's current role is: LEADER
I20260811 22:23:57.378016 31573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:23:57.388968 31573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:23:57.399192 31573 catalog_manager.cc:1347] Generated new cluster ID: 42a60304e22343deac306080a79569d3
I20260811 22:23:57.399340 31573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:23:57.423080 31573 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:23:57.424758 31573 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:23:57.433651 31573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b: Generated new TSK 0
I20260811 22:23:57.434490 31573 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:23:57.442076 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.1:0
--local_ip_for_outbound_sockets=127.29.229.1
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43581
--builtin_ntp_servers=127.29.229.20:41775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:23:57.830716 31592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:57.831127 31592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:57.831300 31592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:57.839033 31592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:57.839187 31592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:57.839375 31592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.1
I20260811 22:23:57.851063 31592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:41775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.229.1
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43581
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.1
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:57.853330 31592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:57.855499 31592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:57.870476 31601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:57.870535 31599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:57.870424 31604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:57.871451 31592 server_base.cc:1034] running on GCE node
I20260811 22:23:57.872237 31592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:57.873394 31592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:57.874650 31592 hybrid_clock.cc:648] HybridClock initialized: now 1786487037874597 us; error 47 us; skew 500 ppm
I20260811 22:23:57.875026 31592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:23:57.878021 31592 webserver.cc:460] Webserver started at http://127.29.229.1:43541/ using document root <none> and password file <none>
I20260811 22:23:57.878834 31592 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:57.878984 31592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:57.879338 31592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:57.882526 31592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-0/data/instance:
uuid: "7046df67768a45ce974a00ae1411336e"
format_stamp: "Formatted at 2026-08-11 22:23:57 on dist-test-slave-jq4t"
server_key: "ce91e049fd4adc6dd1e66bb2f33829b2"
server_key_iv: "e18c6f8550f82f44c8530cad7416490c"
server_key_version: "encryptionkey@0"
I20260811 22:23:57.883389 31592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance:
uuid: "7046df67768a45ce974a00ae1411336e"
format_stamp: "Formatted at 2026-08-11 22:23:57 on dist-test-slave-jq4t"
server_key: "ce91e049fd4adc6dd1e66bb2f33829b2"
server_key_iv: "e18c6f8550f82f44c8530cad7416490c"
server_key_version: "encryptionkey@0"
I20260811 22:23:57.889951 31592 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260811 22:23:57.893954 31611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:57.896071 31592 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260811 22:23:57.896281 31592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-0/wal
uuid: "7046df67768a45ce974a00ae1411336e"
format_stamp: "Formatted at 2026-08-11 22:23:57 on dist-test-slave-jq4t"
server_key: "ce91e049fd4adc6dd1e66bb2f33829b2"
server_key_iv: "e18c6f8550f82f44c8530cad7416490c"
server_key_version: "encryptionkey@0"
I20260811 22:23:57.896577 31592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:57.911305 31592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:57.912215 31592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:57.912616 31592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:57.913810 31592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:57.915704 31592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:57.915855 31592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:57.915997 31592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:57.916088 31592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:57.954245 31592 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.1:43979
I20260811 22:23:57.954345 31785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.1:43979 every 8 connection(s)
I20260811 22:23:57.955741 31592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-0/data/info.pb
I20260811 22:23:57.966333 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 31592
I20260811 22:23:57.966578 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-0/wal/instance
I20260811 22:23:57.967168 30612 external_mini_cluster.cc:1428] Setting key e4bbca63d760f647fbcc4198d9120398
I20260811 22:23:57.970925 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:57.958659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.1:47093 (local address 127.29.229.62:43581)
0811 22:23:57.959208 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:57.959215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:57.960536 (+ 1321us) server_negotiation.cc:407] Connection header received
0811 22:23:57.962385 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:57.962390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:57.962464 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:57.962572 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:57.963692 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:57.964420 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:57.966949 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:57.967651 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:57.967812 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:57.969576 (+ 1764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:57.969581 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:57.969584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:57.969606 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:57.969681 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:57.969700 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:57.969814 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:57.970397 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 22:23:57.970554 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":295,"threads_started":1}
I20260811 22:23:57.971510 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.2:0
--local_ip_for_outbound_sockets=127.29.229.2
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43581
--builtin_ntp_servers=127.29.229.20:41775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:23:57.972514 31789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:57.959207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43581 (local address 127.29.229.1:47093)
0811 22:23:57.960306 (+ 1099us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:57.960350 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:57.962106 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:57.962811 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:57.962819 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:57.962908 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:57.963519 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:57.963537 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:57.964679 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:57.964683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:57.966778 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:57.966793 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:57.968002 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:57.968011 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:57.968313 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:57.969176 (+ 863us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:23:57.969179 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:57.969224 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:23:57.969314 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:57.969863 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:57.969881 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:57.969893 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 22:23:57.970400 (+ 507us) client_negotiation.cc:240] Negotiation successful
0811 22:23:57.970930 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":350,"threads_started":1}
I20260811 22:23:57.974757 31786 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43581
I20260811 22:23:57.975147 31786 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:57.975970 31786 heartbeater.cc:511] Master 127.29.229.62:43581 requested a full tablet report, sending...
I20260811 22:23:57.979005 31489 ts_manager.cc:194] Registered new tserver with Master: 7046df67768a45ce974a00ae1411336e (127.29.229.1:43979)
I20260811 22:23:57.981213 31489 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.229.1:47093
W20260811 22:23:58.357398 31790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:58.357894 31790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:58.358119 31790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:58.367960 31790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:58.368150 31790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:58.368357 31790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.2
I20260811 22:23:58.379112 31790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:41775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.229.2
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43581
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.2
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:58.380612 31790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:58.382601 31790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:58.398171 31799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:58.398278 31798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:58.398260 31802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:58.399147 31790 server_base.cc:1034] running on GCE node
I20260811 22:23:58.399793 31790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:58.400928 31790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:58.402168 31790 hybrid_clock.cc:648] HybridClock initialized: now 1786487038402117 us; error 83 us; skew 500 ppm
I20260811 22:23:58.402462 31790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:23:58.405228 31790 webserver.cc:460] Webserver started at http://127.29.229.2:43165/ using document root <none> and password file <none>
I20260811 22:23:58.405934 31790 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:58.406038 31790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:58.406307 31790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:58.410300 31790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-1/data/instance:
uuid: "da555797de594e36b09904c5a366111d"
format_stamp: "Formatted at 2026-08-11 22:23:58 on dist-test-slave-jq4t"
server_key: "a9bd0723c480faa5fad02722d13e6d5d"
server_key_iv: "82a190a3289ad760d87b3d80800d680c"
server_key_version: "encryptionkey@0"
I20260811 22:23:58.411181 31790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance:
uuid: "da555797de594e36b09904c5a366111d"
format_stamp: "Formatted at 2026-08-11 22:23:58 on dist-test-slave-jq4t"
server_key: "a9bd0723c480faa5fad02722d13e6d5d"
server_key_iv: "82a190a3289ad760d87b3d80800d680c"
server_key_version: "encryptionkey@0"
I20260811 22:23:58.417204 31790 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 22:23:58.421092 31810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:58.422748 31790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 22:23:58.422971 31790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-1/wal
uuid: "da555797de594e36b09904c5a366111d"
format_stamp: "Formatted at 2026-08-11 22:23:58 on dist-test-slave-jq4t"
server_key: "a9bd0723c480faa5fad02722d13e6d5d"
server_key_iv: "82a190a3289ad760d87b3d80800d680c"
server_key_version: "encryptionkey@0"
I20260811 22:23:58.423225 31790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:58.457636 31790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:58.458591 31790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:58.458915 31790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:58.459964 31790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:58.461601 31790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:58.461696 31790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:58.461860 31790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:58.461956 31790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:58.499094 31790 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.2:42861
I20260811 22:23:58.499212 31997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.2:42861 every 8 connection(s)
I20260811 22:23:58.500691 31790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-1/data/info.pb
I20260811 22:23:58.508271 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 31790
I20260811 22:23:58.508636 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-1/wal/instance
I20260811 22:23:58.509370 30612 external_mini_cluster.cc:1428] Setting key 83972d09eeaad08fd0fa0d08fb144777
I20260811 22:23:58.512953 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.229.3:0
--local_ip_for_outbound_sockets=127.29.229.3
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43581
--builtin_ntp_servers=127.29.229.20:41775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:23:58.517779 32001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.504155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:52049 (local address 127.29.229.62:43581)
0811 22:23:58.505013 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:58.505021 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:58.506567 (+ 1546us) server_negotiation.cc:407] Connection header received
0811 22:23:58.508517 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:58.508526 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:58.508770 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:58.508956 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:58.510722 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.511706 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.513762 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.514512 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.514669 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.516371 (+ 1702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:58.516379 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:58.516387 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:58.516414 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:58.516469 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:58.516518 (+ 49us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:58.516616 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:58.517006 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 22:23:58.517205 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":465,"threads_started":1}
I20260811 22:23:58.518417 32002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:58.505261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43581 (local address 127.29.229.2:52049)
0811 22:23:58.506334 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:58.506386 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:58.508144 (+ 1758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:58.509330 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:58.509342 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:58.509515 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:58.510465 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.510491 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.511945 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.511953 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.513578 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:58.513609 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:58.514783 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:58.514789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:58.515016 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:58.516002 (+ 986us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:23:58.516006 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:58.516051 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:23:58.516144 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:58.516633 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:58.516644 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:58.516652 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:23:58.516823 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 22:23:58.517240 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":348,"threads_started":1}
I20260811 22:23:58.520515 31999 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43581
I20260811 22:23:58.520854 31999 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:58.521687 31999 heartbeater.cc:511] Master 127.29.229.62:43581 requested a full tablet report, sending...
I20260811 22:23:58.523397 31489 ts_manager.cc:194] Registered new tserver with Master: da555797de594e36b09904c5a366111d (127.29.229.2:42861)
I20260811 22:23:58.524278 31489 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.229.2:52049
W20260811 22:23:58.897244 32006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:23:58.897593 32006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:23:58.897756 32006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:23:58.905378 32006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:23:58.905514 32006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:23:58.905704 32006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.229.3
I20260811 22:23:58.915185 32006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:41775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.229.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.229.3
--webserver_port=0
--tserver_master_addrs=127.29.229.62:43581
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.229.3
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:23:58.916764 32006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:23:58.918603 32006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:23:58.932171 32016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:58.932173 32018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:23:58.932410 32020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:23:58.933418 32006 server_base.cc:1034] running on GCE node
I20260811 22:23:58.934031 32006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:23:58.935017 32006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:23:58.936298 32006 hybrid_clock.cc:648] HybridClock initialized: now 1786487038936236 us; error 45 us; skew 500 ppm
I20260811 22:23:58.936713 32006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:23:58.939622 32006 webserver.cc:460] Webserver started at http://127.29.229.3:37567/ using document root <none> and password file <none>
I20260811 22:23:58.940371 32006 fs_manager.cc:359] Metadata directory not provided
I20260811 22:23:58.940543 32006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:23:58.940873 32006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:23:58.943867 32006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-2/data/instance:
uuid: "4392d5d471b04c2493d6083940e6688b"
format_stamp: "Formatted at 2026-08-11 22:23:58 on dist-test-slave-jq4t"
server_key: "570649b11669d1ba83cc86d40e4187de"
server_key_iv: "f8713559b6a7cb09e54d2a12bddcef01"
server_key_version: "encryptionkey@0"
I20260811 22:23:58.944777 32006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance:
uuid: "4392d5d471b04c2493d6083940e6688b"
format_stamp: "Formatted at 2026-08-11 22:23:58 on dist-test-slave-jq4t"
server_key: "570649b11669d1ba83cc86d40e4187de"
server_key_iv: "f8713559b6a7cb09e54d2a12bddcef01"
server_key_version: "encryptionkey@0"
I20260811 22:23:58.950682 32006 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 22:23:58.955464 32031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:58.957284 32006 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260811 22:23:58.957481 32006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-2/wal
uuid: "4392d5d471b04c2493d6083940e6688b"
format_stamp: "Formatted at 2026-08-11 22:23:58 on dist-test-slave-jq4t"
server_key: "570649b11669d1ba83cc86d40e4187de"
server_key_iv: "f8713559b6a7cb09e54d2a12bddcef01"
server_key_version: "encryptionkey@0"
I20260811 22:23:58.957748 32006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:23:58.972088 32006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:23:58.973665 32006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:23:58.973954 32006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:23:58.974944 32006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:23:58.976541 32006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:23:58.976647 32006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:58.976727 32006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:23:58.976785 32006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:23:58.984764 31786 heartbeater.cc:503] Master 127.29.229.62:43581 was elected leader, sending a full tablet report...
I20260811 22:23:59.013720 32006 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.3:42501
I20260811 22:23:59.013768 32230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.3:42501 every 8 connection(s)
I20260811 22:23:59.015121 32006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-2/data/info.pb
I20260811 22:23:59.016624 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 32006
I20260811 22:23:59.016819 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-2/wal/instance
I20260811 22:23:59.017246 30612 external_mini_cluster.cc:1428] Setting key 7d2c639b3c43fb90a9e6acfe246badf4
I20260811 22:23:59.030587 32235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.019119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.3:55755 (local address 127.29.229.62:43581)
0811 22:23:59.019597 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:59.019603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:59.021545 (+ 1942us) server_negotiation.cc:407] Connection header received
0811 22:23:59.023402 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:59.023406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:59.023475 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:59.023567 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:59.024756 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.025543 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.027259 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.027730 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.027831 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.029552 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:59.029561 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:59.029566 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:59.029597 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:59.029678 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:59.029696 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:59.029838 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:59.030191 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 22:23:59.030287 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":209,"threads_started":1}
I20260811 22:23:59.032173 32237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.019857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:43581 (local address 127.29.229.3:55755)
0811 22:23:59.021363 (+ 1506us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:59.021411 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:59.023132 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:59.023790 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:59.023798 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:59.023896 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:23:59.024575 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:59.024597 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.025709 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.025713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:59.027115 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:59.027128 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.027992 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.027998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:59.028213 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.029034 (+ 821us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:23:59.029036 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:23:59.029079 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:23:59.029169 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:23:59.029900 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:23:59.029910 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:23:59.029918 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:23:59.030093 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 22:23:59.030423 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":944,"thread_start_us":479,"threads_started":1}
I20260811 22:23:59.033916 32231 heartbeater.cc:347] Connected to a master server at 127.29.229.62:43581
I20260811 22:23:59.034258 32231 heartbeater.cc:464] Registering TS with master...
I20260811 22:23:59.035211 32231 heartbeater.cc:511] Master 127.29.229.62:43581 requested a full tablet report, sending...
I20260811 22:23:59.036715 31488 ts_manager.cc:194] Registered new tserver with Master: 4392d5d471b04c2493d6083940e6688b (127.29.229.3:42501)
I20260811 22:23:59.037650 31488 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.229.3:55755
I20260811 22:23:59.048035 30612 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:23:59.058517 32235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.052407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58634 (local address 127.29.229.62:43581)
0811 22:23:59.052705 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:59.052713 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:59.053157 (+ 444us) server_negotiation.cc:407] Connection header received
0811 22:23:59.053361 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:59.053368 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:59.053438 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:59.053550 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:59.054092 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.054818 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.056182 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.056622 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.056759 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.057820 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:59.057824 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:59.057827 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:59.057845 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:59.057877 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:59.057889 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:59.058022 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:59.058128 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 22:23:59.058221 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260811 22:23:59.062985 31488 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:23:59.070192 31488 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:23:59.102627 32266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.086176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53674 (local address 127.29.229.2:42861)
0811 22:23:59.092337 (+ 6161us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:59.092351 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:59.092397 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:23:59.092840 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:59.092852 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:59.093200 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:59.093411 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:59.094599 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.096240 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.099684 (+ 3444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.101307 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.101462 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.101773 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:59.102040 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 22:23:59.102231 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5910,"thread_start_us":5747,"threads_started":1}
I20260811 22:23:59.102630 32262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.085341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.2:42861 (local address 127.0.0.1:53674)
0811 22:23:59.092193 (+ 6852us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:59.092252 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:59.092494 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:59.093734 (+ 1240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:59.093741 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:59.093776 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:59.094358 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:59.094371 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.096407 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.096412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:59.099412 (+ 3000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:59.099433 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.101516 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.101524 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:59.101754 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.101814 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 22:23:59.101971 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 22:23:59.102198 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6545,"mutex_wait_us":347,"thread_start_us":326,"threads_started":1}
I20260811 22:23:59.102829 32268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.085677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:43979 (local address 127.0.0.1:60100)
0811 22:23:59.093547 (+ 7870us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:59.093571 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:59.093702 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:59.095026 (+ 1324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:59.095031 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:59.095057 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:59.095562 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:59.095575 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.097312 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.097317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:59.099808 (+ 2491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:59.099821 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.102029 (+ 2208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.102036 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:59.102212 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.102243 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 22:23:59.102332 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 22:23:59.102472 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7732,"mutex_wait_us":46,"thread_start_us":8377,"threads_started":1}
I20260811 22:23:59.103718 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.086992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60100 (local address 127.29.229.1:43979)
0811 22:23:59.093955 (+ 6963us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:59.093966 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:59.093995 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:23:59.094099 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:59.094107 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:59.094346 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:59.094645 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:59.095755 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.097164 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.100044 (+ 2880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.101503 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.101629 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.101871 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:59.103164 (+ 1293us) server_negotiation.cc:299] Negotiation successful
0811 22:23:59.103357 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6756,"thread_start_us":6602,"threads_started":1}
I20260811 22:23:59.103922 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.086270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:42501 (local address 127.0.0.1:54756)
0811 22:23:59.095108 (+ 8838us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:59.095131 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:59.095360 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:59.096197 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:59.096200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:59.096219 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:59.096728 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:59.096741 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.098251 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.098256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:59.100421 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:59.100434 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.103022 (+ 2588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.103031 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:59.103215 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.103248 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 22:23:59.103351 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 22:23:59.103494 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8703,"mutex_wait_us":28,"thread_start_us":6989,"threads_started":1}
I20260811 22:23:59.104007 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.086527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54756 (local address 127.29.229.3:42501)
0811 22:23:59.093041 (+ 6514us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:59.093053 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:59.095249 (+ 2196us) server_negotiation.cc:407] Connection header received
0811 22:23:59.095611 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:59.095619 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:59.095835 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:59.095964 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:59.096953 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.098087 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.100716 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.102217 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.102349 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.102638 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:59.103420 (+ 782us) server_negotiation.cc:299] Negotiation successful
0811 22:23:59.103591 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6311,"thread_start_us":6146,"threads_started":1}
I20260811 22:23:59.107076 31897 tablet_service.cc:1467] Processing CreateTablet for tablet 9dc36580155c4cbfa7ec64f1126f9c63 (DEFAULT_TABLE table=test-table [id=072457c0d4974d94bad8b34724be13fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:59.107079 31686 tablet_service.cc:1467] Processing CreateTablet for tablet 9dc36580155c4cbfa7ec64f1126f9c63 (DEFAULT_TABLE table=test-table [id=072457c0d4974d94bad8b34724be13fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:59.108419 31897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dc36580155c4cbfa7ec64f1126f9c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:59.108706 31686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dc36580155c4cbfa7ec64f1126f9c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:59.111843 32117 tablet_service.cc:1467] Processing CreateTablet for tablet 9dc36580155c4cbfa7ec64f1126f9c63 (DEFAULT_TABLE table=test-table [id=072457c0d4974d94bad8b34724be13fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:23:59.113494 32117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dc36580155c4cbfa7ec64f1126f9c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:23:59.122785 32274 tablet_bootstrap.cc:492] T 9dc36580155c4cbfa7ec64f1126f9c63 P 7046df67768a45ce974a00ae1411336e: Bootstrap starting.
I20260811 22:23:59.125334 32276 tablet_bootstrap.cc:492] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d: Bootstrap starting.
I20260811 22:23:59.127151 32274 tablet_bootstrap.cc:654] T 9dc36580155c4cbfa7ec64f1126f9c63 P 7046df67768a45ce974a00ae1411336e: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:59.128553 32274 log.cc:824] T 9dc36580155c4cbfa7ec64f1126f9c63 P 7046df67768a45ce974a00ae1411336e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:59.128984 32277 tablet_bootstrap.cc:492] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b: Bootstrap starting.
I20260811 22:23:59.130674 32276 tablet_bootstrap.cc:654] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:59.132014 32276 log.cc:824] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:59.132176 32277 tablet_bootstrap.cc:654] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b: Neither blocks nor log segments found. Creating new log.
I20260811 22:23:59.132752 32274 tablet_bootstrap.cc:492] T 9dc36580155c4cbfa7ec64f1126f9c63 P 7046df67768a45ce974a00ae1411336e: No bootstrap required, opened a new log
I20260811 22:23:59.133052 32274 ts_tablet_manager.cc:1397] T 9dc36580155c4cbfa7ec64f1126f9c63 P 7046df67768a45ce974a00ae1411336e: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260811 22:23:59.133477 32277 log.cc:824] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:23:59.135305 32276 tablet_bootstrap.cc:492] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d: No bootstrap required, opened a new log
I20260811 22:23:59.135579 32276 ts_tablet_manager.cc:1397] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260811 22:23:59.136157 32277 tablet_bootstrap.cc:492] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b: No bootstrap required, opened a new log
I20260811 22:23:59.136404 32277 ts_tablet_manager.cc:1397] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260811 22:23:59.143848 32274 raft_consensus.cc:348] T 9dc36580155c4cbfa7ec64f1126f9c63 P 7046df67768a45ce974a00ae1411336e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da555797de594e36b09904c5a366111d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42861 } } peers { permanent_uuid: "7046df67768a45ce974a00ae1411336e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43979 } } peers { permanent_uuid: "4392d5d471b04c2493d6083940e6688b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42501 } }
I20260811 22:23:59.144058 32277 raft_consensus.cc:348] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da555797de594e36b09904c5a366111d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42861 } } peers { permanent_uuid: "7046df67768a45ce974a00ae1411336e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43979 } } peers { permanent_uuid: "4392d5d471b04c2493d6083940e6688b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42501 } }
I20260811 22:23:59.144322 32274 raft_consensus.cc:374] T 9dc36580155c4cbfa7ec64f1126f9c63 P 7046df67768a45ce974a00ae1411336e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:59.144428 32274 raft_consensus.cc:729] T 9dc36580155c4cbfa7ec64f1126f9c63 P 7046df67768a45ce974a00ae1411336e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7046df67768a45ce974a00ae1411336e, State: Initialized, Role: FOLLOWER
I20260811 22:23:59.144443 32277 raft_consensus.cc:374] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:59.144629 32277 raft_consensus.cc:729] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4392d5d471b04c2493d6083940e6688b, State: Initialized, Role: FOLLOWER
I20260811 22:23:59.145216 32277 consensus_queue.cc:260] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da555797de594e36b09904c5a366111d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42861 } } peers { permanent_uuid: "7046df67768a45ce974a00ae1411336e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43979 } } peers { permanent_uuid: "4392d5d471b04c2493d6083940e6688b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42501 } }
I20260811 22:23:59.145218 32274 consensus_queue.cc:260] T 9dc36580155c4cbfa7ec64f1126f9c63 P 7046df67768a45ce974a00ae1411336e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da555797de594e36b09904c5a366111d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42861 } } peers { permanent_uuid: "7046df67768a45ce974a00ae1411336e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43979 } } peers { permanent_uuid: "4392d5d471b04c2493d6083940e6688b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42501 } }
I20260811 22:23:59.146411 32231 heartbeater.cc:503] Master 127.29.229.62:43581 was elected leader, sending a full tablet report...
I20260811 22:23:59.146152 32276 raft_consensus.cc:348] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da555797de594e36b09904c5a366111d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42861 } } peers { permanent_uuid: "7046df67768a45ce974a00ae1411336e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43979 } } peers { permanent_uuid: "4392d5d471b04c2493d6083940e6688b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42501 } }
I20260811 22:23:59.146584 32276 raft_consensus.cc:374] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:23:59.146694 32276 raft_consensus.cc:729] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da555797de594e36b09904c5a366111d, State: Initialized, Role: FOLLOWER
I20260811 22:23:59.147413 32276 consensus_queue.cc:260] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da555797de594e36b09904c5a366111d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42861 } } peers { permanent_uuid: "7046df67768a45ce974a00ae1411336e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43979 } } peers { permanent_uuid: "4392d5d471b04c2493d6083940e6688b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42501 } }
I20260811 22:23:59.147984 32274 ts_tablet_manager.cc:1428] T 9dc36580155c4cbfa7ec64f1126f9c63 P 7046df67768a45ce974a00ae1411336e: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260811 22:23:59.148272 32277 ts_tablet_manager.cc:1428] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b: Time spent starting tablet: real 0.012s user 0.005s sys 0.004s
I20260811 22:23:59.150261 31999 heartbeater.cc:503] Master 127.29.229.62:43581 was elected leader, sending a full tablet report...
I20260811 22:23:59.151146 32276 ts_tablet_manager.cc:1428] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d: Time spent starting tablet: real 0.015s user 0.013s sys 0.003s
W20260811 22:23:59.208328 31787 tablet.cc:2369] T 9dc36580155c4cbfa7ec64f1126f9c63 P 7046df67768a45ce974a00ae1411336e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:23:59.253253 32000 tablet.cc:2369] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:23:59.267628 32233 tablet.cc:2369] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:23:59.389242 32285 raft_consensus.cc:482] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:23:59.389561 32285 raft_consensus.cc:504] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da555797de594e36b09904c5a366111d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42861 } } peers { permanent_uuid: "7046df67768a45ce974a00ae1411336e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43979 } } peers { permanent_uuid: "4392d5d471b04c2493d6083940e6688b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42501 } }
I20260811 22:23:59.391415 32285 leader_election.cc:288] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7046df67768a45ce974a00ae1411336e (127.29.229.1:43979), 4392d5d471b04c2493d6083940e6688b (127.29.229.3:42501)
I20260811 22:23:59.399102 32288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.391583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:43979 (local address 127.29.229.2:49967)
0811 22:23:59.392333 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:59.392357 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:59.392542 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:59.392972 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:59.392977 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:59.393004 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:59.393414 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:59.393427 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.394596 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.394602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:59.396856 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:59.396870 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.398263 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.398271 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:59.398446 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.398484 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 22:23:59.398602 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 22:23:59.398756 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":585,"threads_started":1}
I20260811 22:23:59.399500 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.391954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:49967 (local address 127.29.229.1:43979)
0811 22:23:59.392169 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:59.392174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:59.392467 (+ 293us) server_negotiation.cc:407] Connection header received
0811 22:23:59.392736 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:59.392741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:59.392833 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:59.392940 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:59.393577 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.394472 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.397066 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.398057 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.398190 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.398241 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:59.398872 (+ 631us) server_negotiation.cc:299] Negotiation successful
0811 22:23:59.399099 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:23:59.400012 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.392011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.229.2:37987 (local address 127.29.229.3:42501)
0811 22:23:59.392428 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:59.392434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:59.392943 (+ 509us) server_negotiation.cc:407] Connection header received
0811 22:23:59.393194 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:59.393199 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:59.393282 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:59.393394 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:23:59.394037 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.395184 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.397580 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.398615 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.398780 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.398845 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:59.399640 (+ 795us) server_negotiation.cc:299] Negotiation successful
0811 22:23:59.399799 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266}
I20260811 22:23:59.400141 32290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.391976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.3:42501 (local address 127.29.229.2:37987)
0811 22:23:59.392842 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:59.392865 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:23:59.393047 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:23:59.393405 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:23:59.393408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:23:59.393427 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:23:59.393845 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:59.393858 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.395360 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.395365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:59.397362 (+ 1997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:23:59.397378 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.398959 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.398968 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:23:59.399249 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.399298 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 22:23:59.399547 (+ 249us) client_negotiation.cc:240] Negotiation successful
0811 22:23:59.399732 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"mutex_wait_us":1,"thread_start_us":470,"threads_started":1}
I20260811 22:23:59.400969 31719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dc36580155c4cbfa7ec64f1126f9c63" candidate_uuid: "da555797de594e36b09904c5a366111d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7046df67768a45ce974a00ae1411336e" is_pre_election: true
I20260811 22:23:59.401383 32154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dc36580155c4cbfa7ec64f1126f9c63" candidate_uuid: "da555797de594e36b09904c5a366111d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4392d5d471b04c2493d6083940e6688b" is_pre_election: true
I20260811 22:23:59.401706 31719 raft_consensus.cc:2454] T 9dc36580155c4cbfa7ec64f1126f9c63 P 7046df67768a45ce974a00ae1411336e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da555797de594e36b09904c5a366111d in term 0.
I20260811 22:23:59.401912 32154 raft_consensus.cc:2454] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da555797de594e36b09904c5a366111d in term 0.
I20260811 22:23:59.402546 31813 leader_election.cc:302] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7046df67768a45ce974a00ae1411336e, da555797de594e36b09904c5a366111d; no voters:
I20260811 22:23:59.402978 32285 raft_consensus.cc:2781] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:23:59.403141 32285 raft_consensus.cc:482] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:23:59.403254 32285 raft_consensus.cc:3037] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:59.405849 32285 raft_consensus.cc:504] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da555797de594e36b09904c5a366111d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42861 } } peers { permanent_uuid: "7046df67768a45ce974a00ae1411336e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43979 } } peers { permanent_uuid: "4392d5d471b04c2493d6083940e6688b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42501 } }
I20260811 22:23:59.406543 32285 leader_election.cc:288] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [CANDIDATE]: Term 1 election: Requested vote from peers 7046df67768a45ce974a00ae1411336e (127.29.229.1:43979), 4392d5d471b04c2493d6083940e6688b (127.29.229.3:42501)
I20260811 22:23:59.407176 31719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dc36580155c4cbfa7ec64f1126f9c63" candidate_uuid: "da555797de594e36b09904c5a366111d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7046df67768a45ce974a00ae1411336e"
I20260811 22:23:59.407192 32154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dc36580155c4cbfa7ec64f1126f9c63" candidate_uuid: "da555797de594e36b09904c5a366111d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4392d5d471b04c2493d6083940e6688b"
I20260811 22:23:59.407392 31719 raft_consensus.cc:3037] T 9dc36580155c4cbfa7ec64f1126f9c63 P 7046df67768a45ce974a00ae1411336e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:59.407406 32154 raft_consensus.cc:3037] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:23:59.409884 31719 raft_consensus.cc:2454] T 9dc36580155c4cbfa7ec64f1126f9c63 P 7046df67768a45ce974a00ae1411336e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da555797de594e36b09904c5a366111d in term 1.
I20260811 22:23:59.409884 32154 raft_consensus.cc:2454] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da555797de594e36b09904c5a366111d in term 1.
I20260811 22:23:59.410351 31813 leader_election.cc:302] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7046df67768a45ce974a00ae1411336e, da555797de594e36b09904c5a366111d; no voters:
I20260811 22:23:59.410678 32285 raft_consensus.cc:2781] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:23:59.411226 32285 raft_consensus.cc:686] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [term 1 LEADER]: Becoming Leader. State: Replica: da555797de594e36b09904c5a366111d, State: Running, Role: LEADER
I20260811 22:23:59.411720 32285 consensus_queue.cc:237] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da555797de594e36b09904c5a366111d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42861 } } peers { permanent_uuid: "7046df67768a45ce974a00ae1411336e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43979 } } peers { permanent_uuid: "4392d5d471b04c2493d6083940e6688b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42501 } }
I20260811 22:23:59.417027 31486 catalog_manager.cc:5495] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d reported cstate change: term changed from 0 to 1, leader changed from <none> to da555797de594e36b09904c5a366111d (127.29.229.2). New cstate: current_term: 1 leader_uuid: "da555797de594e36b09904c5a366111d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da555797de594e36b09904c5a366111d" member_type: VOTER last_known_addr { host: "127.29.229.2" port: 42861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7046df67768a45ce974a00ae1411336e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4392d5d471b04c2493d6083940e6688b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42501 } health_report { overall_health: UNKNOWN } } }
I20260811 22:23:59.474730 32266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.468061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53684 (local address 127.29.229.2:42861)
0811 22:23:59.468248 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:59.468252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:59.468270 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:23:59.468399 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:59.468403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:59.468581 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:59.468717 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:23:59.469241 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.470128 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.472258 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.472783 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.472898 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.473511 (+ 613us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:23:59.474068 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:23:59.474186 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:59.474421 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 22:23:59.474499 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:23:59.483366 32154 raft_consensus.cc:1261] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b [term 1 FOLLOWER]: Refusing update from remote peer da555797de594e36b09904c5a366111d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:23:59.483371 31719 raft_consensus.cc:1261] T 9dc36580155c4cbfa7ec64f1126f9c63 P 7046df67768a45ce974a00ae1411336e [term 1 FOLLOWER]: Refusing update from remote peer da555797de594e36b09904c5a366111d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:23:59.484431 32292 consensus_queue.cc:1035] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4392d5d471b04c2493d6083940e6688b" member_type: VOTER last_known_addr { host: "127.29.229.3" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:23:59.484908 32285 consensus_queue.cc:1035] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7046df67768a45ce974a00ae1411336e" member_type: VOTER last_known_addr { host: "127.29.229.1" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:23:59.497018 32296 mvcc.cc:205] Tried to move back new op lower bound from 7317450913713799168 to 7317450913434595328. Current Snapshot: MvccSnapshot[applied={T|T < 7317450913713799168}]
I20260811 22:23:59.503348 32300 mvcc.cc:205] Tried to move back new op lower bound from 7317450913713799168 to 7317450913434595328. Current Snapshot: MvccSnapshot[applied={T|T < 7317450913713799168}]
I20260811 22:23:59.524806 31486 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:23:59.525134 31486 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:23:59.529065 31486 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a117c717163475ba318823631ffec9b: Sending DeleteTablet for 3 replicas of tablet 9dc36580155c4cbfa7ec64f1126f9c63
I20260811 22:23:59.530474 31686 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dc36580155c4cbfa7ec64f1126f9c63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:59 UTC) from {username='slave'} at 127.0.0.1:60100
I20260811 22:23:59.530475 32117 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dc36580155c4cbfa7ec64f1126f9c63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:59 UTC) from {username='slave'} at 127.0.0.1:54756
I20260811 22:23:59.530719 31897 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dc36580155c4cbfa7ec64f1126f9c63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:23:59 UTC) from {username='slave'} at 127.0.0.1:53674
I20260811 22:23:59.538555 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 31592
I20260811 22:23:59.542155 32314 tablet_replica.cc:330] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d: stopping tablet replica
I20260811 22:23:59.542984 32314 raft_consensus.cc:2229] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:23:59.544203 32314 raft_consensus.cc:2258] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:59.545687 32312 tablet_replica.cc:330] T 9dc36580155c4cbfa7ec64f1126f9c63 P 7046df67768a45ce974a00ae1411336e: stopping tablet replica
I20260811 22:23:59.546121 32312 raft_consensus.cc:2229] T 9dc36580155c4cbfa7ec64f1126f9c63 P 7046df67768a45ce974a00ae1411336e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:23:59.546631 32312 raft_consensus.cc:2258] T 9dc36580155c4cbfa7ec64f1126f9c63 P 7046df67768a45ce974a00ae1411336e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:59.547385 32313 tablet_replica.cc:330] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b: stopping tablet replica
I20260811 22:23:59.547752 32313 raft_consensus.cc:2229] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:23:59.548214 32313 raft_consensus.cc:2258] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:23:59.550624 32313 ts_tablet_manager.cc:1905] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:59.550635 32314 ts_tablet_manager.cc:1905] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:59.554390 32312 ts_tablet_manager.cc:1905] T 9dc36580155c4cbfa7ec64f1126f9c63 P 7046df67768a45ce974a00ae1411336e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:23:59.559504 32314 ts_tablet_manager.cc:1918] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:23:59.559942 32314 log.cc:1196] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-1/wal/wals/9dc36580155c4cbfa7ec64f1126f9c63
I20260811 22:23:59.561542 32314 ts_tablet_manager.cc:1939] T 9dc36580155c4cbfa7ec64f1126f9c63 P da555797de594e36b09904c5a366111d: Deleting consensus metadata
I20260811 22:23:59.563719 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.539549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60114 (local address 127.29.229.1:43979)
0811 22:23:59.539758 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:59.539764 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:59.554692 (+ 14928us) server_negotiation.cc:407] Connection header received
0811 22:23:59.555221 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:59.555227 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:59.555347 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:59.555435 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:59.556141 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.557383 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.559380 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.560072 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.560180 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.561805 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:59.561820 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:59.561838 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:59.562682 (+ 844us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:59.562744 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:59.562775 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:59.562869 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:59.563194 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 22:23:59.563324 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:23:59.563874 31463 catalog_manager.cc:4841] TS da555797de594e36b09904c5a366111d (127.29.229.2:42861): tablet 9dc36580155c4cbfa7ec64f1126f9c63 (table test-table [id=072457c0d4974d94bad8b34724be13fd]) successfully deleted
I20260811 22:23:59.565021 31779 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:59.569334 32313 ts_tablet_manager.cc:1918] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:23:59.569541 32313 log.cc:1196] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b: Deleting WAL directory at /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486685119861-30612-0/minicluster-data/ts-2/wal/wals/9dc36580155c4cbfa7ec64f1126f9c63
I20260811 22:23:59.570050 32313 ts_tablet_manager.cc:1939] T 9dc36580155c4cbfa7ec64f1126f9c63 P 4392d5d471b04c2493d6083940e6688b: Deleting consensus metadata
I20260811 22:23:59.571841 31464 catalog_manager.cc:4841] TS 4392d5d471b04c2493d6083940e6688b (127.29.229.3:42501): tablet 9dc36580155c4cbfa7ec64f1126f9c63 (table test-table [id=072457c0d4974d94bad8b34724be13fd]) successfully deleted
I20260811 22:23:59.665732 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 31592
I20260811 22:23:59.702387 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 31790
W20260811 22:23:59.702342 31465 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.229.1:43979 (error 108)
I20260811 22:23:59.705119 32325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.703736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:43979 (local address 127.0.0.1:60130)
0811 22:23:59.704698 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:59.704876 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:43979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":835,"thread_start_us":759,"threads_started":1}
W20260811 22:23:59.705427 31465 catalog_manager.cc:4584] TS 7046df67768a45ce974a00ae1411336e (127.29.229.1:43979): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9dc36580155c4cbfa7ec64f1126f9c63: Network error: Client connection negotiation failed: client connection to 127.29.229.1:43979: connect: Connection refused (error 111)
I20260811 22:23:59.711977 32266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.703707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53692 (local address 127.29.229.2:42861)
0811 22:23:59.703931 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:59.703943 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:59.704702 (+ 759us) server_negotiation.cc:407] Connection header received
0811 22:23:59.704786 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:59.704791 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:59.704910 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:59.705016 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:59.705685 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.706681 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.707986 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.708445 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.708641 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.710321 (+ 1680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:59.710331 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:59.710347 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:59.711090 (+ 743us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:59.711146 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:59.711168 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:59.711326 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:59.711495 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 22:23:59.711609 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:23:59.712845 31989 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:59.759531 32325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.758917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:43979 (local address 127.0.0.1:60136)
0811 22:23:59.759220 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:59.759332 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:43979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 22:23:59.761219 32325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.760672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:43979 (local address 127.0.0.1:60138)
0811 22:23:59.760919 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:59.761024 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:43979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 22:23:59.822062 32325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.821544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:43979 (local address 127.0.0.1:60140)
0811 22:23:59.821782 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:59.821879 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:43979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 22:23:59.823226 32325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.822822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:43979 (local address 127.0.0.1:60154)
0811 22:23:59.823010 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:59.823081 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:43979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 22:23:59.836133 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 31790
I20260811 22:23:59.867214 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 32006
I20260811 22:23:59.874573 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.868059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54772 (local address 127.29.229.3:42501)
0811 22:23:59.868231 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 22:23:59.868235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:23:59.868253 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:23:59.868358 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:23:59.868362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:23:59.868443 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:23:59.868593 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:23:59.869029 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.869810 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.871373 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:23:59.871790 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:23:59.871939 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:23:59.873151 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:23:59.873160 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:23:59.873175 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:23:59.873764 (+ 589us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:23:59.873811 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:23:59.873834 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:23:59.873965 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 22:23:59.874103 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 22:23:59.874219 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:23:59.875617 32224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:23:59.904129 32325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.903549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:43979 (local address 127.0.0.1:60170)
0811 22:23:59.903816 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:59.903930 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:43979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 22:23:59.905377 32325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:23:59.904893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:43979 (local address 127.0.0.1:60174)
0811 22:23:59.905125 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 22:23:59.905210 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:43979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260811 22:23:59.971283 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 32006
I20260811 22:23:59.998459 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 31426
I20260811 22:23:59.999703 31542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:24:00.121403 32325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:00.120094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.1:43979 (local address 127.0.0.1:60184)
0811 22:24:00.120990 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:00.121133 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.1:43979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":706}
I20260811 22:24:00.121588 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 31426
2026-08-11T22:24:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3418 ms)
[----------] 6 tests from EncryptionPolicyTest (26043 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:00 dist-test-slave-jq4t krb5kdc[32334](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:00 dist-test-slave-jq4t krb5kdc[32334](info): set up 2 sockets
Aug 11 22:24:00 dist-test-slave-jq4t krb5kdc[32334](info): commencing operation
krb5kdc: starting...
W20260811 22:24:01.571319 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.393s user 0.000s sys 0.062s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:01 dist-test-slave-jq4t krb5kdc[32334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:24:01.985097 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40227
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:02.365622 32356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:02.365934 32356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:02.366118 32356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:02.374617 32356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:02.374737 32356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:02.374847 32356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:02.374980 32356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:02.375092 32356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:02.386731 32356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40227
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:02.388165 32356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:02.389576 32356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:02.401894 32375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:02.402009 32371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:02.401968 32373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:02.403237 32356 server_base.cc:1034] running on GCE node
I20260811 22:24:02.404125 32356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:02.405588 32356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:02.406787 32356 hybrid_clock.cc:648] HybridClock initialized: now 1786487042406745 us; error 39 us; skew 500 ppm
I20260811 22:24:02.407160 32356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:02 dist-test-slave-jq4t krb5kdc[32334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487042, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:02.414359 32356 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:24:02.416064 32356 webserver.cc:460] Webserver started at http://127.0.0.1:38699/ using document root <none> and password file <none>
I20260811 22:24:02.416909 32356 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:02.417060 32356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:02.417419 32356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:02.420650 32356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "840cdfd2c4974b0dadd797f23d786dc1"
format_stamp: "Formatted at 2026-08-11 22:24:02 on dist-test-slave-jq4t"
server_key: "527fd673a73f296b3b060685a1067fec"
server_key_iv: "112a9b8a8e7c7cd8f479560446d44c07"
server_key_version: "encryptionkey@0"
I20260811 22:24:02.421556 32356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "840cdfd2c4974b0dadd797f23d786dc1"
format_stamp: "Formatted at 2026-08-11 22:24:02 on dist-test-slave-jq4t"
server_key: "527fd673a73f296b3b060685a1067fec"
server_key_iv: "112a9b8a8e7c7cd8f479560446d44c07"
server_key_version: "encryptionkey@0"
I20260811 22:24:02.427790 32356 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 22:24:02.431725 32385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:02.433215 32356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:24:02.433415 32356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "840cdfd2c4974b0dadd797f23d786dc1"
format_stamp: "Formatted at 2026-08-11 22:24:02 on dist-test-slave-jq4t"
server_key: "527fd673a73f296b3b060685a1067fec"
server_key_iv: "112a9b8a8e7c7cd8f479560446d44c07"
server_key_version: "encryptionkey@0"
I20260811 22:24:02.433622 32356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:02.462740 32356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:02.467682 32356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:02.468080 32356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:02.488971 32356 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40227
I20260811 22:24:02.489024 32482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40227 every 8 connection(s)
I20260811 22:24:02.490428 32356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:02.494369 32483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:02.495728 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 32356
I20260811 22:24:02.496436 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:02.497255 30612 external_mini_cluster.cc:1428] Setting key 7855fc598d150341112c2caf8b2c55c6
I20260811 22:24:02.508087 32483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1: Bootstrap starting.
I20260811 22:24:02.512424 32483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:02.513952 32483 log.cc:824] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:24:02 dist-test-slave-jq4t krb5kdc[32334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:24:02.518199 32483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1: No bootstrap required, opened a new log
I20260811 22:24:02.527252 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:02.499728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49745 (local address 127.0.0.1:40227)
0811 22:24:02.500769 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:02.500781 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:02.500836 (+ 55us) server_negotiation.cc:407] Connection header received
0811 22:24:02.503650 (+ 2814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:02.503674 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:02.504147 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:02.504603 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:02.506638 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:02.508044 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:02.510042 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:02.510822 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:02.510986 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:02.517518 (+ 6532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:02.517532 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:02.517633 (+ 101us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:02.517706 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:02.522150 (+ 4444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:02.523350 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:02.523358 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:02.523365 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:02.523569 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:02.524126 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:02.524130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:02.524132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:02.524847 (+ 715us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:02.525110 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:02.525693 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 22:24:02.526060 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":809,"thread_start_us":441,"threads_started":1}
I20260811 22:24:02.528414 32483 raft_consensus.cc:348] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "840cdfd2c4974b0dadd797f23d786dc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40227 } }
I20260811 22:24:02.528882 32483 raft_consensus.cc:374] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:02.528996 32483 raft_consensus.cc:729] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 840cdfd2c4974b0dadd797f23d786dc1, State: Initialized, Role: FOLLOWER
I20260811 22:24:02.529610 32483 consensus_queue.cc:260] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "840cdfd2c4974b0dadd797f23d786dc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40227 } }
I20260811 22:24:02.529906 32483 raft_consensus.cc:388] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:02.530087 32483 raft_consensus.cc:482] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:02.530290 32483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:02.531829 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:02.534413 32483 raft_consensus.cc:504] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "840cdfd2c4974b0dadd797f23d786dc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40227 } }
I20260811 22:24:02.535142 32483 leader_election.cc:302] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 840cdfd2c4974b0dadd797f23d786dc1; no voters:
I20260811 22:24:02.536150 32497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:02.536207 32483 leader_election.cc:288] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:02.537014 32497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1 [term 1 LEADER]: Becoming Leader. State: Replica: 840cdfd2c4974b0dadd797f23d786dc1, State: Running, Role: LEADER
I20260811 22:24:02.537833 32497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "840cdfd2c4974b0dadd797f23d786dc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40227 } }
I20260811 22:24:02.540469 32483 sys_catalog.cc:564] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:02.541967 32500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 840cdfd2c4974b0dadd797f23d786dc1. Latest consensus state: current_term: 1 leader_uuid: "840cdfd2c4974b0dadd797f23d786dc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "840cdfd2c4974b0dadd797f23d786dc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40227 } } }
I20260811 22:24:02.542222 32501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "840cdfd2c4974b0dadd797f23d786dc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "840cdfd2c4974b0dadd797f23d786dc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40227 } } }
I20260811 22:24:02.542567 32500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:02.542567 32501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:02.556638 32509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:02.562459 32509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:02.565224 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:02.541304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49333 (local address 127.0.0.1:40227)
0811 22:24:02.541566 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:02.541576 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:02.541679 (+ 103us) server_negotiation.cc:407] Connection header received
0811 22:24:02.541868 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:02.541874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:02.541997 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:02.542140 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:02.545378 (+ 3238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:02.546986 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:02.549589 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:02.550652 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:02.550812 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:02.554160 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:02.554171 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:02.554180 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:02.554236 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:02.559402 (+ 5166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:02.560816 (+ 1414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:02.560822 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:02.560826 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:02.561036 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:02.561781 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:02.561789 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:02.561795 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:02.562359 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:02.562719 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:02.564359 (+ 1640us) server_negotiation.cc:299] Negotiation successful
0811 22:24:02.564670 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260811 22:24:02.577471 32509 catalog_manager.cc:1347] Generated new cluster ID: 86a2706fa044421eb7d0f50993d25092
I20260811 22:24:02.577611 32509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:02.610770 32509 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:02.612205 32509 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:02.622437 32509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 840cdfd2c4974b0dadd797f23d786dc1: Generated new TSK 0
I20260811 22:24:02.623473 32509 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:02.644176 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 32356
I20260811 22:24:02.645844 32469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:24:02.752923 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 32356
W20260811 22:24:02.783089 32331 connection.cc:535] client connection to 127.0.0.1:40227 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40227): Connection reset by peer (error 104)
2026-08-11T22:24:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2624 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:02 dist-test-slave-jq4t krb5kdc[32539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:02 dist-test-slave-jq4t krb5kdc[32539](info): set up 2 sockets
Aug 11 22:24:02 dist-test-slave-jq4t krb5kdc[32539](info): commencing operation
krb5kdc: starting...
W20260811 22:24:04.167086 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.364s user 0.001s sys 0.066s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:04 dist-test-slave-jq4t krb5kdc[32539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:24:04.554236 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39895
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:04.948767 32562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:04.949100 32562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:04.949203 32562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:04.957146 32562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:04.957341 32562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:04.957419 32562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:04.957501 32562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:04.957545 32562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:04.968639 32562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39895
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:04.970000 32562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:04.971539 32562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:04.984238 32572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:04.984308 32575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:04.984367 32573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:04.985754 32562 server_base.cc:1034] running on GCE node
I20260811 22:24:04.986742 32562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:04.988409 32562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:04.989711 32562 hybrid_clock.cc:648] HybridClock initialized: now 1786487044989671 us; error 37 us; skew 500 ppm
I20260811 22:24:04.990128 32562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:04 dist-test-slave-jq4t krb5kdc[32539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487044, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:04.993654 32562 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:24:04.996140 32562 webserver.cc:460] Webserver started at http://127.0.0.1:37813/ using document root <none> and password file <none>
I20260811 22:24:04.997274 32562 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:04.997422 32562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:04.997782 32562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:05.001765 32562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "922261d719bc49dfb7767dbf0a6899da"
format_stamp: "Formatted at 2026-08-11 22:24:04 on dist-test-slave-jq4t"
server_key: "bd8747092074c7b37fd1a35a11faba92"
server_key_iv: "709bf708a7d4704895ea7781a58bf3f4"
server_key_version: "encryptionkey@0"
I20260811 22:24:05.002738 32562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "922261d719bc49dfb7767dbf0a6899da"
format_stamp: "Formatted at 2026-08-11 22:24:04 on dist-test-slave-jq4t"
server_key: "bd8747092074c7b37fd1a35a11faba92"
server_key_iv: "709bf708a7d4704895ea7781a58bf3f4"
server_key_version: "encryptionkey@0"
I20260811 22:24:05.009912 32562 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 22:24:05.014158 32586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:05.016072 32562 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:24:05.016316 32562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "922261d719bc49dfb7767dbf0a6899da"
format_stamp: "Formatted at 2026-08-11 22:24:04 on dist-test-slave-jq4t"
server_key: "bd8747092074c7b37fd1a35a11faba92"
server_key_iv: "709bf708a7d4704895ea7781a58bf3f4"
server_key_version: "encryptionkey@0"
I20260811 22:24:05.016623 32562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:05.029265 32562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:05.034168 32562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:05.034509 32562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:05.057696 32562 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39895
I20260811 22:24:05.057745 32674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39895 every 8 connection(s)
I20260811 22:24:05.059360 32562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:05.063763 32675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:05.066473 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 32562
I20260811 22:24:05.066666 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:05.067252 30612 external_mini_cluster.cc:1428] Setting key 97ad6d230a5eed9955fb89703bd090b8
I20260811 22:24:05.077021 32675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da: Bootstrap starting.
I20260811 22:24:05.080634 32675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:05.082118 32675 log.cc:824] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:05.083803 32680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.069399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54911 (local address 127.0.0.1:39895)
0811 22:24:05.070257 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:05.070270 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:05.070314 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:24:05.072082 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:05.072106 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:05.072625 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:05.073018 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:05.073690 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.075708 (+ 2018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.078082 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.078885 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.079131 (+ 246us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.080778 (+ 1647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:05.080789 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:05.080809 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:05.081616 (+ 807us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:05.081739 (+ 123us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:05.081778 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:05.081985 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:05.082331 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 22:24:05.082519 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":373,"threads_started":1}
I20260811 22:24:05.086341 32675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da: No bootstrap required, opened a new log
I20260811 22:24:05.088349 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:05.095112 32675 raft_consensus.cc:348] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922261d719bc49dfb7767dbf0a6899da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39895 } }
I20260811 22:24:05.095671 32675 raft_consensus.cc:374] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:05.095778 32675 raft_consensus.cc:729] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 922261d719bc49dfb7767dbf0a6899da, State: Initialized, Role: FOLLOWER
I20260811 22:24:05.096580 32675 consensus_queue.cc:260] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922261d719bc49dfb7767dbf0a6899da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39895 } }
I20260811 22:24:05.096853 32675 raft_consensus.cc:388] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:05.097028 32675 raft_consensus.cc:482] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:05.097234 32675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:05.101305 32675 raft_consensus.cc:504] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922261d719bc49dfb7767dbf0a6899da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39895 } }
I20260811 22:24:05.101622 32680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:05.092905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41805 (local address 127.0.0.1:39895)
0811 22:24:05.093108 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:05.093116 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:05.093579 (+ 463us) server_negotiation.cc:407] Connection header received
0811 22:24:05.093742 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:05.093748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:05.093857 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:05.093982 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:05.094544 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.095697 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.097856 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:05.098635 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:05.098789 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:05.100508 (+ 1719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:05.100515 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:05.100520 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:05.100545 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:05.100630 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:05.100653 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:05.100776 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:05.101133 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 22:24:05.101202 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:24:05.102015 32675 leader_election.cc:302] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 922261d719bc49dfb7767dbf0a6899da; no voters:
I20260811 22:24:05.102717 32675 leader_election.cc:288] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:05.102952 32689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:05.103976 32689 raft_consensus.cc:686] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da [term 1 LEADER]: Becoming Leader. State: Replica: 922261d719bc49dfb7767dbf0a6899da, State: Running, Role: LEADER
I20260811 22:24:05.104563 32689 consensus_queue.cc:237] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922261d719bc49dfb7767dbf0a6899da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39895 } }
I20260811 22:24:05.105093 32675 sys_catalog.cc:564] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:05.107441 32690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "922261d719bc49dfb7767dbf0a6899da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922261d719bc49dfb7767dbf0a6899da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39895 } } }
I20260811 22:24:05.107882 32690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:05.107656 32691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 922261d719bc49dfb7767dbf0a6899da. Latest consensus state: current_term: 1 leader_uuid: "922261d719bc49dfb7767dbf0a6899da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922261d719bc49dfb7767dbf0a6899da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39895 } } }
I20260811 22:24:05.108093 32691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:05.110231 32699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:05.117861 32699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:05.127535 32699 catalog_manager.cc:1347] Generated new cluster ID: 47051baa7efd47a4a8b805d2edb5549d
I20260811 22:24:05.127674 32699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:05.145009 32699 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:05.146857 32699 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:05.153728 32699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 922261d719bc49dfb7767dbf0a6899da: Generated new TSK 0
I20260811 22:24:05.154587 32699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:05.163604 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 32562
W20260811 22:24:05.165000 32667 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54911
W20260811 22:24:05.166450 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 32562: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:24:05.166659 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 32562
W20260811 22:24:05.193779 32535 connection.cc:535] client connection to 127.0.0.1:39895 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39895): Connection reset by peer (error 104)
2026-08-11T22:24:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2410 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:05 dist-test-slave-jq4t krb5kdc[32726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:05 dist-test-slave-jq4t krb5kdc[32726](info): set up 2 sockets
Aug 11 22:24:05 dist-test-slave-jq4t krb5kdc[32726](info): commencing operation
krb5kdc: starting...
W20260811 22:24:06.594813 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.381s user 0.001s sys 0.064s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:06 dist-test-slave-jq4t krb5kdc[32726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:24:06.995888 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40029
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:07.401257 32750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:07.401576 32750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:07.401731 32750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:07.409176 32750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:07.409302 32750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:07.409420 32750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:07.409505 32750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:07.409584 32750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:07.420338 32750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40029
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:07.421774 32750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:07.423126 32750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:07.435331 32756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:07.435293 32758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:07.435247 32761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:07.436156 32750 server_base.cc:1034] running on GCE node
I20260811 22:24:07.437052 32750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:07.438462 32750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:07.439697 32750 hybrid_clock.cc:648] HybridClock initialized: now 1786487047439653 us; error 41 us; skew 500 ppm
I20260811 22:24:07.440086 32750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:07 dist-test-slave-jq4t krb5kdc[32726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487047, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:07.447165 32750 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:24:07.448853 32750 webserver.cc:460] Webserver started at http://127.0.0.1:41749/ using document root <none> and password file <none>
I20260811 22:24:07.449581 32750 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:07.449734 32750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:07.450073 32750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:07.454459 32750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "f926e916727a4afcb94e076fb5f61774"
format_stamp: "Formatted at 2026-08-11 22:24:07 on dist-test-slave-jq4t"
server_key: "b2382ae08374d9f53134f2182f821236"
server_key_iv: "ef973a02b6bf312ecb8ef0da9d3a24bd"
server_key_version: "encryptionkey@0"
I20260811 22:24:07.455552 32750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "f926e916727a4afcb94e076fb5f61774"
format_stamp: "Formatted at 2026-08-11 22:24:07 on dist-test-slave-jq4t"
server_key: "b2382ae08374d9f53134f2182f821236"
server_key_iv: "ef973a02b6bf312ecb8ef0da9d3a24bd"
server_key_version: "encryptionkey@0"
I20260811 22:24:07.461335 32750 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 22:24:07.465169 301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:07.466670 32750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 22:24:07.466885 32750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "f926e916727a4afcb94e076fb5f61774"
format_stamp: "Formatted at 2026-08-11 22:24:07 on dist-test-slave-jq4t"
server_key: "b2382ae08374d9f53134f2182f821236"
server_key_iv: "ef973a02b6bf312ecb8ef0da9d3a24bd"
server_key_version: "encryptionkey@0"
I20260811 22:24:07.467115 32750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:07.473475 32750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:07.477823 32750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:07.478142 32750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:07.501092 32750 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40029
I20260811 22:24:07.501113 403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40029 every 8 connection(s)
I20260811 22:24:07.502599 32750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:07.503047 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 32750
I20260811 22:24:07.503238 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:07.503847 30612 external_mini_cluster.cc:1428] Setting key 981200caa95ef3df1b1ed83205a8381c
I20260811 22:24:07.507570 404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:07.512655 407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:07.505871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35351 (local address 127.0.0.1:40029)
0811 22:24:07.506906 (+ 1035us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:07.506921 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:07.507258 (+ 337us) server_negotiation.cc:407] Connection header received
0811 22:24:07.509219 (+ 1961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:07.509242 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:07.509557 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:07.509869 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:07.511045 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:07.511052 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:07.511064 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:07.511088 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:07.511161 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:07.511187 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:07.511361 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:07.511474 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 22:24:07.511671 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":732,"thread_start_us":431,"threads_started":1}
I20260811 22:24:07.517448 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:07.521890 404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774: Bootstrap starting.
I20260811 22:24:07.525343 404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:07.526558 404 log.cc:824] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:07.527323 407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:07.522088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50745 (local address 127.0.0.1:40029)
0811 22:24:07.522284 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:07.522291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:07.525084 (+ 2793us) server_negotiation.cc:407] Connection header received
0811 22:24:07.525243 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:07.525248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:07.525328 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:07.525452 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:07.526452 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:07.526458 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:07.526462 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:07.526492 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:07.526552 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:07.526579 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:07.526710 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:07.526867 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 22:24:07.526928 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:24:07.529768 404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774: No bootstrap required, opened a new log
I20260811 22:24:07.537261 404 raft_consensus.cc:348] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926e916727a4afcb94e076fb5f61774" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40029 } }
I20260811 22:24:07.537781 404 raft_consensus.cc:374] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:07.537885 404 raft_consensus.cc:729] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f926e916727a4afcb94e076fb5f61774, State: Initialized, Role: FOLLOWER
I20260811 22:24:07.538484 404 consensus_queue.cc:260] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926e916727a4afcb94e076fb5f61774" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40029 } }
I20260811 22:24:07.538697 404 raft_consensus.cc:388] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:07.538801 404 raft_consensus.cc:482] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:07.538929 404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:07.541544 404 raft_consensus.cc:504] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926e916727a4afcb94e076fb5f61774" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40029 } }
I20260811 22:24:07.542039 404 leader_election.cc:302] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f926e916727a4afcb94e076fb5f61774; no voters:
I20260811 22:24:07.542583 404 leader_election.cc:288] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:07.542759 418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:07.543519 418 raft_consensus.cc:686] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774 [term 1 LEADER]: Becoming Leader. State: Replica: f926e916727a4afcb94e076fb5f61774, State: Running, Role: LEADER
I20260811 22:24:07.544240 418 consensus_queue.cc:237] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926e916727a4afcb94e076fb5f61774" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40029 } }
I20260811 22:24:07.544852 404 sys_catalog.cc:564] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:07.547639 419 sys_catalog.cc:455] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f926e916727a4afcb94e076fb5f61774" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926e916727a4afcb94e076fb5f61774" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40029 } } }
I20260811 22:24:07.547775 420 sys_catalog.cc:455] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f926e916727a4afcb94e076fb5f61774. Latest consensus state: current_term: 1 leader_uuid: "f926e916727a4afcb94e076fb5f61774" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f926e916727a4afcb94e076fb5f61774" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40029 } } }
I20260811 22:24:07.548118 419 sys_catalog.cc:458] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:07.548178 420 sys_catalog.cc:458] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:07.549903 426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:07.554931 426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:07.564595 426 catalog_manager.cc:1347] Generated new cluster ID: 8a4303740df64385b595185e0d1ea34e
I20260811 22:24:07.564747 426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:07.591789 426 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:07.592305 426 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:07.602198 426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f926e916727a4afcb94e076fb5f61774: Generated new TSK 0
I20260811 22:24:07.603112 426 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:07.613933 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 32750
W20260811 22:24:07.615125 397 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35351
W20260811 22:24:07.615875 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 32750: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:24:07.616030 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 32750
2026-08-11T22:24:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2448 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:07 dist-test-slave-jq4t krb5kdc[454](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:07 dist-test-slave-jq4t krb5kdc[454](info): set up 2 sockets
Aug 11 22:24:07 dist-test-slave-jq4t krb5kdc[454](info): commencing operation
krb5kdc: starting...
W20260811 22:24:09.030867 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.369s user 0.000s sys 0.066s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:09 dist-test-slave-jq4t krb5kdc[454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:24:09.435669 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39527
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:09.811694 475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:09.812071 475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:09.812250 475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:09.819681 475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:09.819859 475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:09.819976 475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:09.820101 475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:09.820189 475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:09.830710 475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:09.832247 475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:09.833689 475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:09.847461 486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:09.847577 488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:09.847385 490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:09.848840 475 server_base.cc:1034] running on GCE node
I20260811 22:24:09.850064 475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:09.852113 475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:09.853437 475 hybrid_clock.cc:648] HybridClock initialized: now 1786487049853300 us; error 97 us; skew 500 ppm
I20260811 22:24:09.853999 475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:09 dist-test-slave-jq4t krb5kdc[454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487049, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:09.858693 475 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:24:09.860809 475 webserver.cc:460] Webserver started at http://127.0.0.1:44997/ using document root <none> and password file <none>
I20260811 22:24:09.861598 475 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:09.861759 475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:09.862102 475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:09.865443 475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "3b3cb2721e0b47ef9eeec5967c9b724b"
format_stamp: "Formatted at 2026-08-11 22:24:09 on dist-test-slave-jq4t"
server_key: "4b0e1d0596bfa923875fe17cc8b6153d"
server_key_iv: "f4eee111ce136392226c9de4e14f4718"
server_key_version: "encryptionkey@0"
I20260811 22:24:09.866279 475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "3b3cb2721e0b47ef9eeec5967c9b724b"
format_stamp: "Formatted at 2026-08-11 22:24:09 on dist-test-slave-jq4t"
server_key: "4b0e1d0596bfa923875fe17cc8b6153d"
server_key_iv: "f4eee111ce136392226c9de4e14f4718"
server_key_version: "encryptionkey@0"
I20260811 22:24:09.871829 475 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 22:24:09.875586 502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:09.876948 475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:24:09.877168 475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "3b3cb2721e0b47ef9eeec5967c9b724b"
format_stamp: "Formatted at 2026-08-11 22:24:09 on dist-test-slave-jq4t"
server_key: "4b0e1d0596bfa923875fe17cc8b6153d"
server_key_iv: "f4eee111ce136392226c9de4e14f4718"
server_key_version: "encryptionkey@0"
I20260811 22:24:09.877380 475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:09.904639 475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:09.910291 475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:09.910653 475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:09.931970 475 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39527
I20260811 22:24:09.932019 592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39527 every 8 connection(s)
I20260811 22:24:09.933405 475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:09.936050 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 475
I20260811 22:24:09.936245 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:09.936813 30612 external_mini_cluster.cc:1428] Setting key 6124372fbc958309ad75cb56e29c3f17
I20260811 22:24:09.938074 593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 22:24:09 dist-test-slave-jq4t krb5kdc[454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:24:09.952883 593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b: Bootstrap starting.
I20260811 22:24:09.956463 593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:09.957629 593 log.cc:824] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:09.960853 593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b: No bootstrap required, opened a new log
I20260811 22:24:09.961942 601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:09.939137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58865 (local address 127.0.0.1:39527)
0811 22:24:09.940006 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:09.940019 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:09.940064 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:24:09.941957 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:09.941979 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:09.942376 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:09.942823 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:09.944381 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:09.946443 (+ 2062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:09.947896 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:09.948669 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:09.948892 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:09.953487 (+ 4595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:09.953496 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:09.953512 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:09.953557 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:09.957082 (+ 3525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:09.957959 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:09.957967 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:09.957974 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:09.958256 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:09.958728 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:09.958733 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:09.958737 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:09.959433 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:09.959771 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:09.960284 (+ 513us) server_negotiation.cc:299] Negotiation successful
0811 22:24:09.960811 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":394,"threads_started":1}
I20260811 22:24:09.966081 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:09.969539 593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3cb2721e0b47ef9eeec5967c9b724b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39527 } }
I20260811 22:24:09.969924 593 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:09.970049 593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b3cb2721e0b47ef9eeec5967c9b724b, State: Initialized, Role: FOLLOWER
I20260811 22:24:09.970666 593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3cb2721e0b47ef9eeec5967c9b724b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39527 } }
I20260811 22:24:09.971001 593 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:09.971189 593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:09.971431 593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:09.974066 593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3cb2721e0b47ef9eeec5967c9b724b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39527 } }
I20260811 22:24:09.974714 593 leader_election.cc:302] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b3cb2721e0b47ef9eeec5967c9b724b; no voters:
I20260811 22:24:09.975440 593 leader_election.cc:288] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:09.975711 616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:09.977658 593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:09.978085 616 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b [term 1 LEADER]: Becoming Leader. State: Replica: 3b3cb2721e0b47ef9eeec5967c9b724b, State: Running, Role: LEADER
I20260811 22:24:09.978731 616 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3cb2721e0b47ef9eeec5967c9b724b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39527 } }
I20260811 22:24:09.984902 618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b3cb2721e0b47ef9eeec5967c9b724b. Latest consensus state: current_term: 1 leader_uuid: "3b3cb2721e0b47ef9eeec5967c9b724b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3cb2721e0b47ef9eeec5967c9b724b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39527 } } }
I20260811 22:24:09.985664 618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:09.985509 617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b3cb2721e0b47ef9eeec5967c9b724b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3cb2721e0b47ef9eeec5967c9b724b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39527 } } }
I20260811 22:24:09.985889 617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:09.987190 628 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:09.993042 601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:09.971130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45563 (local address 127.0.0.1:39527)
0811 22:24:09.971322 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:09.971328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:09.971824 (+ 496us) server_negotiation.cc:407] Connection header received
0811 22:24:09.971978 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:09.971983 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:09.972070 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:09.972186 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:09.974248 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:09.975418 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:09.977774 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:09.978492 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:09.978644 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:09.982504 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:09.982514 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:09.982526 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:09.982584 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:09.987166 (+ 4582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:09.988198 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:09.988208 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:09.988219 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:09.988810 (+ 591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:09.989674 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:09.989683 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:09.989688 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:09.990212 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:09.991589 (+ 1377us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:09.991927 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 22:24:09.992431 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:24:09.999017 628 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:10.012025 628 catalog_manager.cc:1347] Generated new cluster ID: 1ab12aa8cc764636a2e6dcc7b1f76953
I20260811 22:24:10.012205 628 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:10.028323 628 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:10.029747 628 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:10.039889 628 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b3cb2721e0b47ef9eeec5967c9b724b: Generated new TSK 0
I20260811 22:24:10.041026 628 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:10.049216 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 475
I20260811 22:24:10.050526 583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:24:10.149010 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 475
2026-08-11T22:24:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2537 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:10 dist-test-slave-jq4t krb5kdc[648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:10 dist-test-slave-jq4t krb5kdc[648](info): set up 2 sockets
Aug 11 22:24:10 dist-test-slave-jq4t krb5kdc[648](info): commencing operation
krb5kdc: starting...
W20260811 22:24:11.578379 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.379s user 0.004s sys 0.061s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:11 dist-test-slave-jq4t krb5kdc[648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:24:11.982443 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43919
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:12.364439 667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:12.364909 667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:12.365083 667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:12.373332 667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:12.373499 667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:12.373617 667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:12.373745 667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:12.373826 667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:12.384588 667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:12.386255 667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:12.387750 667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:12.400453 675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:12.400566 674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:12.400548 678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:12.401656 667 server_base.cc:1034] running on GCE node
I20260811 22:24:12.402541 667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:12.404038 667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:12.405277 667 hybrid_clock.cc:648] HybridClock initialized: now 1786487052405238 us; error 34 us; skew 500 ppm
I20260811 22:24:12.405695 667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:12 dist-test-slave-jq4t krb5kdc[648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487052, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:12.409355 667 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:24:12.411398 667 webserver.cc:460] Webserver started at http://127.0.0.1:34391/ using document root <none> and password file <none>
I20260811 22:24:12.412176 667 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:12.412339 667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:12.412724 667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:12.415975 667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "ae6fcd72547c4fd98537cb6bd47fe45a"
format_stamp: "Formatted at 2026-08-11 22:24:12 on dist-test-slave-jq4t"
server_key: "3b666c38425beefaa413004b903b44f7"
server_key_iv: "ca5f61c5263d99969fef32637bc9c7ac"
server_key_version: "encryptionkey@0"
I20260811 22:24:12.416875 667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "ae6fcd72547c4fd98537cb6bd47fe45a"
format_stamp: "Formatted at 2026-08-11 22:24:12 on dist-test-slave-jq4t"
server_key: "3b666c38425beefaa413004b903b44f7"
server_key_iv: "ca5f61c5263d99969fef32637bc9c7ac"
server_key_version: "encryptionkey@0"
I20260811 22:24:12.422855 667 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 22:24:12.426738 690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:12.428094 667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:24:12.428310 667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "ae6fcd72547c4fd98537cb6bd47fe45a"
format_stamp: "Formatted at 2026-08-11 22:24:12 on dist-test-slave-jq4t"
server_key: "3b666c38425beefaa413004b903b44f7"
server_key_iv: "ca5f61c5263d99969fef32637bc9c7ac"
server_key_version: "encryptionkey@0"
I20260811 22:24:12.428550 667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:12.460814 667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:12.466040 667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:12.466361 667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:12.488602 667 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43919
I20260811 22:24:12.488605 779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43919 every 8 connection(s)
I20260811 22:24:12.490094 667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:12.493172 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 667
I20260811 22:24:12.493487 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:12.494181 30612 external_mini_cluster.cc:1428] Setting key 114c46126871c4d08e392a61ba116edd
I20260811 22:24:12.494436 783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:12.512293 783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a: Bootstrap starting.
I20260811 22:24:12.512432 787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:12.496260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46985 (local address 127.0.0.1:43919)
0811 22:24:12.497175 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:12.497190 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:12.497241 (+ 51us) server_negotiation.cc:407] Connection header received
0811 22:24:12.499665 (+ 2424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:12.499695 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:12.500272 (+ 577us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:12.500981 (+ 709us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:12.501577 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:12.504564 (+ 2987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:12.506374 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:12.507491 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:12.507720 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:12.509346 (+ 1626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:12.509363 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:12.509384 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:12.510188 (+ 804us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:12.510290 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:12.510323 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:12.510446 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:12.510722 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 22:24:12.511052 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":397,"threads_started":1}
I20260811 22:24:12.515825 783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:12.516433 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:12.517105 783 log.cc:824] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:12.520217 783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a: No bootstrap required, opened a new log
I20260811 22:24:12.529086 787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:12.521551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52641 (local address 127.0.0.1:43919)
0811 22:24:12.521788 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:12.521794 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:12.522193 (+ 399us) server_negotiation.cc:407] Connection header received
0811 22:24:12.522450 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:12.522454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:12.522521 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:12.522615 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:12.523261 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:12.524021 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:12.526049 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:12.526828 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:12.526964 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:12.528252 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:12.528260 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:12.528264 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:12.528282 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:12.528326 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:12.528341 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:12.528422 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:12.528608 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 22:24:12.528692 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:24:12.530864 783 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae6fcd72547c4fd98537cb6bd47fe45a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43919 } }
I20260811 22:24:12.531304 783 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:12.531423 783 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae6fcd72547c4fd98537cb6bd47fe45a, State: Initialized, Role: FOLLOWER
I20260811 22:24:12.532014 783 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae6fcd72547c4fd98537cb6bd47fe45a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43919 } }
I20260811 22:24:12.532289 783 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:12.532445 783 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:12.532675 783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:12.535277 783 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae6fcd72547c4fd98537cb6bd47fe45a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43919 } }
I20260811 22:24:12.535866 783 leader_election.cc:302] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae6fcd72547c4fd98537cb6bd47fe45a; no voters:
I20260811 22:24:12.536661 783 leader_election.cc:288] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:12.536795 798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:12.537861 798 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a [term 1 LEADER]: Becoming Leader. State: Replica: ae6fcd72547c4fd98537cb6bd47fe45a, State: Running, Role: LEADER
I20260811 22:24:12.538450 798 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae6fcd72547c4fd98537cb6bd47fe45a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43919 } }
I20260811 22:24:12.539054 783 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:12.542351 801 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae6fcd72547c4fd98537cb6bd47fe45a. Latest consensus state: current_term: 1 leader_uuid: "ae6fcd72547c4fd98537cb6bd47fe45a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae6fcd72547c4fd98537cb6bd47fe45a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43919 } } }
I20260811 22:24:12.542201 799 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae6fcd72547c4fd98537cb6bd47fe45a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae6fcd72547c4fd98537cb6bd47fe45a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43919 } } }
I20260811 22:24:12.542786 801 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:12.542819 799 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:12.544590 806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:12.551123 806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:12.562516 806 catalog_manager.cc:1347] Generated new cluster ID: cad8acfefe6c4e8eb08aaa7f851e1b28
I20260811 22:24:12.562846 806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:12.592990 806 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:12.594978 806 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:12.615197 806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae6fcd72547c4fd98537cb6bd47fe45a: Generated new TSK 0
I20260811 22:24:12.616338 806 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:12.626803 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 667
W20260811 22:24:12.628134 772 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46985
W20260811 22:24:12.628955 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 667: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:24:12.629132 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 667
2026-08-11T22:24:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2485 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:12 dist-test-slave-jq4t krb5kdc[841](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:12 dist-test-slave-jq4t krb5kdc[841](info): set up 2 sockets
Aug 11 22:24:12 dist-test-slave-jq4t krb5kdc[841](info): commencing operation
krb5kdc: starting...
W20260811 22:24:14.073593 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.389s user 0.001s sys 0.063s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:14 dist-test-slave-jq4t krb5kdc[841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:24:14.473169 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45063
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:14.863832 871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:14.864145 871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:14.864243 871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:14.871842 871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:14.871984 871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:14.872035 871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:14.872099 871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:14.872144 871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:14.882376 871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:14.883735 871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:14.885170 871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:14.898424 880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:14.898470 884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:14.898517 881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:14.899477 871 server_base.cc:1034] running on GCE node
I20260811 22:24:14.900305 871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:14.902660 871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:14.904054 871 hybrid_clock.cc:648] HybridClock initialized: now 1786487054903955 us; error 94 us; skew 500 ppm
I20260811 22:24:14.904541 871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:14 dist-test-slave-jq4t krb5kdc[841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487054, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:14.908840 871 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:24:14.910671 871 webserver.cc:460] Webserver started at http://127.0.0.1:41243/ using document root <none> and password file <none>
I20260811 22:24:14.911947 871 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:14.912328 871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:14.912765 871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:14.916289 871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "7d3deade1bc54be7b6f8fce0745faf42"
format_stamp: "Formatted at 2026-08-11 22:24:14 on dist-test-slave-jq4t"
server_key: "0028baadcb7a356ea4f72d3c7c074912"
server_key_iv: "747d5ea8b7ee600ab9a92a92958068da"
server_key_version: "encryptionkey@0"
I20260811 22:24:14.917238 871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "7d3deade1bc54be7b6f8fce0745faf42"
format_stamp: "Formatted at 2026-08-11 22:24:14 on dist-test-slave-jq4t"
server_key: "0028baadcb7a356ea4f72d3c7c074912"
server_key_iv: "747d5ea8b7ee600ab9a92a92958068da"
server_key_version: "encryptionkey@0"
I20260811 22:24:14.922960 871 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 22:24:14.926872 895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:14.928282 871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:24:14.928553 871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "7d3deade1bc54be7b6f8fce0745faf42"
format_stamp: "Formatted at 2026-08-11 22:24:14 on dist-test-slave-jq4t"
server_key: "0028baadcb7a356ea4f72d3c7c074912"
server_key_iv: "747d5ea8b7ee600ab9a92a92958068da"
server_key_version: "encryptionkey@0"
I20260811 22:24:14.928779 871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:14.936865 871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:14.941830 871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:14.942162 871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:14.964984 871 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45063
I20260811 22:24:14.965035 982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45063 every 8 connection(s)
I20260811 22:24:14.966461 871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:14.970710 983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:14.974474 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 871
I20260811 22:24:14.974677 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:14.975243 30612 external_mini_cluster.cc:1428] Setting key 2a029087e1501f448edd0716562d6338
I20260811 22:24:14.983312 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:14.977233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45765 (local address 127.0.0.1:45063)
0811 22:24:14.978086 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:14.978098 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:14.978139 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:24:14.979800 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:14.979823 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:14.980179 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:14.980593 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:14.981694 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:14.981701 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:14.981717 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:14.981740 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:14.981797 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:14.981820 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:14.982024 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:14.982122 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 22:24:14.982337 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":388,"threads_started":1}
I20260811 22:24:14.984148 983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42: Bootstrap starting.
I20260811 22:24:14.988131 983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:14.988427 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:14.989423 983 log.cc:824] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:14.992906 983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42: No bootstrap required, opened a new log
I20260811 22:24:14.996829 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:14.994173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40351 (local address 127.0.0.1:45063)
0811 22:24:14.994405 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:14.994413 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:14.994663 (+ 250us) server_negotiation.cc:407] Connection header received
0811 22:24:14.994894 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:14.994899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:14.994950 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:14.995093 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:14.996095 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:14.996101 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:14.996106 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:14.996131 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:14.996177 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:14.996193 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:14.996333 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:14.996429 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 22:24:14.996538 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 22:24:15.001717 983 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3deade1bc54be7b6f8fce0745faf42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45063 } }
I20260811 22:24:15.002086 983 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:15.002154 983 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d3deade1bc54be7b6f8fce0745faf42, State: Initialized, Role: FOLLOWER
I20260811 22:24:15.002732 983 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3deade1bc54be7b6f8fce0745faf42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45063 } }
I20260811 22:24:15.002960 983 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:15.003149 983 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:15.003355 983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:15.006902 983 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3deade1bc54be7b6f8fce0745faf42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45063 } }
I20260811 22:24:15.007653 983 leader_election.cc:302] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d3deade1bc54be7b6f8fce0745faf42; no voters:
I20260811 22:24:15.008602 983 leader_election.cc:288] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:15.008785 1003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:15.009687 1003 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42 [term 1 LEADER]: Becoming Leader. State: Replica: 7d3deade1bc54be7b6f8fce0745faf42, State: Running, Role: LEADER
I20260811 22:24:15.010528 1003 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3deade1bc54be7b6f8fce0745faf42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45063 } }
I20260811 22:24:15.011451 983 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:15.014618 1005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d3deade1bc54be7b6f8fce0745faf42. Latest consensus state: current_term: 1 leader_uuid: "7d3deade1bc54be7b6f8fce0745faf42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3deade1bc54be7b6f8fce0745faf42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45063 } } }
I20260811 22:24:15.014485 1006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d3deade1bc54be7b6f8fce0745faf42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3deade1bc54be7b6f8fce0745faf42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45063 } } }
I20260811 22:24:15.015105 1005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:15.015105 1006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:15.016412 1015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:15.022965 1015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:15.033414 1015 catalog_manager.cc:1347] Generated new cluster ID: 70b1b3dd9d91404fa9450d24f2b849b6
I20260811 22:24:15.033566 1015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:15.059757 1015 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:15.060253 1015 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:15.066521 1015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d3deade1bc54be7b6f8fce0745faf42: Generated new TSK 0
I20260811 22:24:15.067349 1015 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:15.076176 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 871
W20260811 22:24:15.077512 977 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45765
W20260811 22:24:15.078342 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 871: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:24:15.078518 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 871
2026-08-11T22:24:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:15 dist-test-slave-jq4t krb5kdc[1038](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:15 dist-test-slave-jq4t krb5kdc[1038](info): set up 2 sockets
Aug 11 22:24:15 dist-test-slave-jq4t krb5kdc[1038](info): commencing operation
krb5kdc: starting...
W20260811 22:24:16.525894 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.395s user 0.003s sys 0.062s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:16 dist-test-slave-jq4t krb5kdc[1038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:24:16.945089 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33505
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:17.336120 1060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:17.336634 1060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:17.336809 1060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:17.347720 1060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:17.347911 1060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:17.348012 1060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:17.348133 1060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:17.348210 1060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:17.365844 1060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:17.367776 1060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:17.369421 1060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:17.386422 1067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:17.387099 1071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:17.387255 1066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:17.387913 1060 server_base.cc:1034] running on GCE node
I20260811 22:24:17.388972 1060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:17.390626 1060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:17.391873 1060 hybrid_clock.cc:648] HybridClock initialized: now 1786487057391794 us; error 66 us; skew 500 ppm
I20260811 22:24:17.392287 1060 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:17 dist-test-slave-jq4t krb5kdc[1038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487057, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:17.396100 1060 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:24:17.398201 1060 webserver.cc:460] Webserver started at http://127.0.0.1:37351/ using document root <none> and password file <none>
I20260811 22:24:17.399199 1060 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:17.399350 1060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:17.399710 1060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:17.404024 1060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "25664042a97644d980646b31e3b5d904"
format_stamp: "Formatted at 2026-08-11 22:24:17 on dist-test-slave-jq4t"
server_key: "c386c4881194315f2da6e07a6076bfef"
server_key_iv: "db59f57339542ef1f1409cef97692ad0"
server_key_version: "encryptionkey@0"
I20260811 22:24:17.404992 1060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "25664042a97644d980646b31e3b5d904"
format_stamp: "Formatted at 2026-08-11 22:24:17 on dist-test-slave-jq4t"
server_key: "c386c4881194315f2da6e07a6076bfef"
server_key_iv: "db59f57339542ef1f1409cef97692ad0"
server_key_version: "encryptionkey@0"
I20260811 22:24:17.411187 1060 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 22:24:17.415153 1083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:17.416925 1060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 22:24:17.417130 1060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "25664042a97644d980646b31e3b5d904"
format_stamp: "Formatted at 2026-08-11 22:24:17 on dist-test-slave-jq4t"
server_key: "c386c4881194315f2da6e07a6076bfef"
server_key_iv: "db59f57339542ef1f1409cef97692ad0"
server_key_version: "encryptionkey@0"
I20260811 22:24:17.417348 1060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:17.431545 1060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:17.436254 1060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:17.436671 1060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:17.457963 1060 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33505
I20260811 22:24:17.458003 1162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33505 every 8 connection(s)
I20260811 22:24:17.459451 1060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:17.463505 1164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:17.467501 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 1060
I20260811 22:24:17.467751 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:17.468276 30612 external_mini_cluster.cc:1428] Setting key e9aceea23bbe1b75078cca504a5c95c5
I20260811 22:24:17.477913 1164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904: Bootstrap starting.
I20260811 22:24:17.482023 1164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:17.483095 1164 log.cc:824] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:17.487474 1164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904: No bootstrap required, opened a new log
Aug 11 22:24:17 dist-test-slave-jq4t krb5kdc[1038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:24:17.498533 1164 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25664042a97644d980646b31e3b5d904" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33505 } }
I20260811 22:24:17.499030 1164 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:17.499171 1164 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25664042a97644d980646b31e3b5d904, State: Initialized, Role: FOLLOWER
I20260811 22:24:17.499989 1170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:17.470590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:39619 (local address 127.0.0.1:33505)
0811 22:24:17.471430 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:17.471442 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:17.471491 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:24:17.473501 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:17.473527 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:17.473991 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:17.474340 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:17.476309 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:17.477793 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:17.479923 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:17.480747 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:17.480889 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:17.489705 (+ 8816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:17.489719 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:17.489742 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:17.489809 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:17.494528 (+ 4719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:17.495666 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:17.495677 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:17.495694 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:17.495945 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:17.496531 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:17.496536 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:17.496540 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:17.497369 (+ 829us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:17.497709 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:17.498166 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 22:24:17.498531 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":403,"threads_started":1}
I20260811 22:24:17.499869 1164 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25664042a97644d980646b31e3b5d904" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33505 } }
I20260811 22:24:17.500185 1164 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:17.500355 1164 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:17.500617 1164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:17.504022 1164 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25664042a97644d980646b31e3b5d904" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33505 } }
I20260811 22:24:17.504457 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:17.504678 1164 leader_election.cc:302] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25664042a97644d980646b31e3b5d904; no voters:
I20260811 22:24:17.505352 1164 leader_election.cc:288] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:17.505496 1177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:17.508018 1164 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:17.509594 1177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904 [term 1 LEADER]: Becoming Leader. State: Replica: 25664042a97644d980646b31e3b5d904, State: Running, Role: LEADER
I20260811 22:24:17.510861 1177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25664042a97644d980646b31e3b5d904" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33505 } }
I20260811 22:24:17.515209 1179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25664042a97644d980646b31e3b5d904. Latest consensus state: current_term: 1 leader_uuid: "25664042a97644d980646b31e3b5d904" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25664042a97644d980646b31e3b5d904" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33505 } } }
I20260811 22:24:17.515627 1179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:17.515695 1178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25664042a97644d980646b31e3b5d904" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25664042a97644d980646b31e3b5d904" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33505 } } }
I20260811 22:24:17.516178 1178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:17.525578 1193 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:17.531718 1193 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:17.535002 1170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:17.510398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:52389 (local address 127.0.0.1:33505)
0811 22:24:17.510642 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:17.510651 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:17.510716 (+ 65us) server_negotiation.cc:407] Connection header received
0811 22:24:17.510984 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:17.510991 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:17.511104 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:17.511215 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:17.513521 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:17.515220 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:17.517451 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:17.518435 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:17.518605 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:17.523332 (+ 4727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:17.523341 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:17.523349 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:17.523415 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:17.528429 (+ 5014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:17.529975 (+ 1546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:17.529983 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:17.529990 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:17.530208 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:17.531233 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:17.531242 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:17.531248 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:17.531842 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:17.532233 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:17.534096 (+ 1863us) server_negotiation.cc:299] Negotiation successful
0811 22:24:17.534386 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 22:24:17.542917 1193 catalog_manager.cc:1347] Generated new cluster ID: 5d1484e70a3a4308a3f1523dedada4c5
I20260811 22:24:17.543053 1193 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:17.571951 1193 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:17.574080 1193 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:17.594004 1193 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25664042a97644d980646b31e3b5d904: Generated new TSK 0
I20260811 22:24:17.595108 1193 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:17.609407 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 1060
I20260811 22:24:17.610783 1156 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:24:17.711616 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 1060
W20260811 22:24:17.741474 1035 connection.cc:535] client connection to 127.0.0.1:33505 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33505): Connection reset by peer (error 104)
2026-08-11T22:24:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2634 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:17 dist-test-slave-jq4t krb5kdc[1224](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:17 dist-test-slave-jq4t krb5kdc[1224](info): set up 2 sockets
Aug 11 22:24:17 dist-test-slave-jq4t krb5kdc[1224](info): commencing operation
krb5kdc: starting...
W20260811 22:24:19.141909 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.378s user 0.000s sys 0.071s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:19 dist-test-slave-jq4t krb5kdc[1224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:24:19.553045 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39445
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:19.934808 1245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:19.935145 1245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:19.935245 1245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:19.943207 1245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:19.943364 1245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:19.943414 1245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:19.943481 1245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:19.943528 1245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:19.954134 1245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:19.955472 1245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:19.956828 1245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:19.969834 1253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:19.969887 1262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:19.969909 1255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:19.971266 1245 server_base.cc:1034] running on GCE node
I20260811 22:24:19.972128 1245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:19.973634 1245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:19.974871 1245 hybrid_clock.cc:648] HybridClock initialized: now 1786487059974827 us; error 30 us; skew 500 ppm
I20260811 22:24:19.975206 1245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:19 dist-test-slave-jq4t krb5kdc[1224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487059, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:19.979154 1245 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:24:19.981470 1245 webserver.cc:460] Webserver started at http://127.0.0.1:46823/ using document root <none> and password file <none>
I20260811 22:24:19.982385 1245 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:19.982566 1245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:19.982985 1245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:19.986498 1245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "5a32c65e530d4f2aa496a4d78a1c45bb"
format_stamp: "Formatted at 2026-08-11 22:24:19 on dist-test-slave-jq4t"
server_key: "5545d84e44aa31876f8180ae2e0bce49"
server_key_iv: "4766d1cc9bede941ca4012cdb5b5b500"
server_key_version: "encryptionkey@0"
I20260811 22:24:19.987367 1245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "5a32c65e530d4f2aa496a4d78a1c45bb"
format_stamp: "Formatted at 2026-08-11 22:24:19 on dist-test-slave-jq4t"
server_key: "5545d84e44aa31876f8180ae2e0bce49"
server_key_iv: "4766d1cc9bede941ca4012cdb5b5b500"
server_key_version: "encryptionkey@0"
I20260811 22:24:19.993372 1245 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 22:24:19.997361 1274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:19.999168 1245 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260811 22:24:19.999388 1245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "5a32c65e530d4f2aa496a4d78a1c45bb"
format_stamp: "Formatted at 2026-08-11 22:24:19 on dist-test-slave-jq4t"
server_key: "5545d84e44aa31876f8180ae2e0bce49"
server_key_iv: "4766d1cc9bede941ca4012cdb5b5b500"
server_key_version: "encryptionkey@0"
I20260811 22:24:19.999621 1245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:20.022977 1245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:20.029332 1245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:20.029743 1245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:20.056346 1245 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39445
I20260811 22:24:20.056376 1369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39445 every 8 connection(s)
I20260811 22:24:20.058020 1245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:20.062348 1370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:20.068169 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 1245
I20260811 22:24:20.068343 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:20.068866 30612 external_mini_cluster.cc:1428] Setting key 7f6ff2646e801bad45abaa840421e463
I20260811 22:24:20.075685 1370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb: Bootstrap starting.
I20260811 22:24:20.079823 1370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:20.081112 1370 log.cc:824] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:20.085462 1370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb: No bootstrap required, opened a new log
I20260811 22:24:20.086156 1376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:20.071152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:43033 (local address 127.0.0.1:39445)
0811 22:24:20.072028 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:20.072041 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:20.072087 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:24:20.073996 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:20.074023 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:20.074461 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:20.074818 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:20.075759 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:20.077658 (+ 1899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:20.079798 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:20.081040 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:20.081307 (+ 267us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:20.083082 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:20.083097 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:20.083138 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:20.083908 (+ 770us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:20.084018 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:20.084052 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:20.084223 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:20.084799 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 22:24:20.085034 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":402,"threads_started":1}
I20260811 22:24:20.091629 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:20.094653 1370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a32c65e530d4f2aa496a4d78a1c45bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39445 } }
I20260811 22:24:20.095144 1370 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:20.095279 1370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a32c65e530d4f2aa496a4d78a1c45bb, State: Initialized, Role: FOLLOWER
I20260811 22:24:20.095988 1370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a32c65e530d4f2aa496a4d78a1c45bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39445 } }
I20260811 22:24:20.096238 1370 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:20.096395 1370 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:20.096640 1370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:20.099243 1370 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a32c65e530d4f2aa496a4d78a1c45bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39445 } }
I20260811 22:24:20.099809 1370 leader_election.cc:302] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a32c65e530d4f2aa496a4d78a1c45bb; no voters:
I20260811 22:24:20.100581 1370 leader_election.cc:288] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:20.100693 1387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:20.103469 1370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:20.104759 1387 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb [term 1 LEADER]: Becoming Leader. State: Replica: 5a32c65e530d4f2aa496a4d78a1c45bb, State: Running, Role: LEADER
I20260811 22:24:20.105237 1376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:20.097565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:36781 (local address 127.0.0.1:39445)
0811 22:24:20.097874 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:20.097884 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:20.098411 (+ 527us) server_negotiation.cc:407] Connection header received
0811 22:24:20.098638 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:20.098642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:20.098711 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:20.098818 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:20.099347 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:20.100200 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:20.102104 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:20.102691 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:20.102827 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:20.104141 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:20.104146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:20.104149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:20.104171 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:20.104232 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:20.104249 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:20.104420 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:20.104817 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 22:24:20.104894 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:24:20.105525 1387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a32c65e530d4f2aa496a4d78a1c45bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39445 } }
I20260811 22:24:20.109616 1389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a32c65e530d4f2aa496a4d78a1c45bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a32c65e530d4f2aa496a4d78a1c45bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39445 } } }
I20260811 22:24:20.109977 1389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:20.111649 1402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:20.111491 1393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a32c65e530d4f2aa496a4d78a1c45bb. Latest consensus state: current_term: 1 leader_uuid: "5a32c65e530d4f2aa496a4d78a1c45bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a32c65e530d4f2aa496a4d78a1c45bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39445 } } }
I20260811 22:24:20.111814 1393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:20.117583 1402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:20.127923 1402 catalog_manager.cc:1347] Generated new cluster ID: 6cc92bdb863749b1a6bd00ebc0ee7e1b
I20260811 22:24:20.128080 1402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:20.144719 1402 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:20.145970 1402 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:20.155957 1402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a32c65e530d4f2aa496a4d78a1c45bb: Generated new TSK 0
I20260811 22:24:20.156903 1402 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:20.164713 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 1245
W20260811 22:24:20.166278 1362 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.48:43033
W20260811 22:24:20.167323 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 1245: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:24:20.167495 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 1245
W20260811 22:24:20.192294 1220 connection.cc:535] client connection to 127.0.0.1:39445 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39445): Connection reset by peer (error 104)
2026-08-11T22:24:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2448 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:20 dist-test-slave-jq4t krb5kdc[1427](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:20 dist-test-slave-jq4t krb5kdc[1427](info): set up 2 sockets
Aug 11 22:24:20 dist-test-slave-jq4t krb5kdc[1427](info): commencing operation
krb5kdc: starting...
W20260811 22:24:21.592329 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.378s user 0.000s sys 0.067s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:21 dist-test-slave-jq4t krb5kdc[1427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:24:22.004408 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36521
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:22.377242 1456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:22.377549 1456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:22.377729 1456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:22.385371 1456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:22.385495 1456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:22.385607 1456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:22.385713 1456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:22.385800 1456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:22.395938 1456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:22.397384 1456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:22.398780 1456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:22.412220 1464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:22.412195 1468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:22.412139 1465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:22.413036 1456 server_base.cc:1034] running on GCE node
I20260811 22:24:22.413833 1456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:22.415455 1456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:22.416656 1456 hybrid_clock.cc:648] HybridClock initialized: now 1786487062416615 us; error 45 us; skew 500 ppm
I20260811 22:24:22.417049 1456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:22 dist-test-slave-jq4t krb5kdc[1427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487062, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:22.420565 1456 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:24:22.422209 1456 webserver.cc:460] Webserver started at http://127.0.0.1:46273/ using document root <none> and password file <none>
I20260811 22:24:22.422914 1456 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:22.423063 1456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:22.423380 1456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:22.426580 1456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "8789570fb8de4cb1956e3c8a169aa4f3"
format_stamp: "Formatted at 2026-08-11 22:24:22 on dist-test-slave-jq4t"
server_key: "c893f0e4cefd16f3383c8d2a47450d90"
server_key_iv: "9aef50dfbe3a8a6394dfeeab8853b64d"
server_key_version: "encryptionkey@0"
I20260811 22:24:22.427371 1456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "8789570fb8de4cb1956e3c8a169aa4f3"
format_stamp: "Formatted at 2026-08-11 22:24:22 on dist-test-slave-jq4t"
server_key: "c893f0e4cefd16f3383c8d2a47450d90"
server_key_iv: "9aef50dfbe3a8a6394dfeeab8853b64d"
server_key_version: "encryptionkey@0"
I20260811 22:24:22.433638 1456 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260811 22:24:22.437604 1480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:22.439064 1456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:24:22.439371 1456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "8789570fb8de4cb1956e3c8a169aa4f3"
format_stamp: "Formatted at 2026-08-11 22:24:22 on dist-test-slave-jq4t"
server_key: "c893f0e4cefd16f3383c8d2a47450d90"
server_key_iv: "9aef50dfbe3a8a6394dfeeab8853b64d"
server_key_version: "encryptionkey@0"
I20260811 22:24:22.439621 1456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:22.445896 1456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:22.450714 1456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:22.451064 1456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:22.472885 1456 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36521
I20260811 22:24:22.472945 1568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36521 every 8 connection(s)
I20260811 22:24:22.474380 1456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:22.476346 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 1456
I20260811 22:24:22.476583 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:22.477166 30612 external_mini_cluster.cc:1428] Setting key e2b9dacee4d73cd91216a7006d6f27ba
I20260811 22:24:22.479370 1569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:22.485772 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:22.479082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:39183 (local address 127.0.0.1:36521)
0811 22:24:22.479915 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:22.479926 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:22.479968 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:24:22.481695 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:22.481718 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:22.482066 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:22.482487 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:22.483884 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:22.483890 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:22.483903 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:22.483919 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:22.484015 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:22.484037 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:22.484289 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:22.484441 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 22:24:22.484652 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":400,"threads_started":1}
I20260811 22:24:22.490429 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:22.492904 1569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3: Bootstrap starting.
I20260811 22:24:22.496775 1569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:22.497869 1569 log.cc:824] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:22.501096 1569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3: No bootstrap required, opened a new log
I20260811 22:24:22.504276 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:22.495446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:40293 (local address 127.0.0.1:36521)
0811 22:24:22.495720 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:22.495727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:22.495977 (+ 250us) server_negotiation.cc:407] Connection header received
0811 22:24:22.496276 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:22.496281 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:22.496343 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:22.496461 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:22.503512 (+ 7051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:22.503517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:22.503521 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:22.503553 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:22.503613 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:22.503631 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:22.503781 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:22.503909 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 22:24:22.503986 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 22:24:22.509248 1569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8789570fb8de4cb1956e3c8a169aa4f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36521 } }
I20260811 22:24:22.509697 1569 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:22.509809 1569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8789570fb8de4cb1956e3c8a169aa4f3, State: Initialized, Role: FOLLOWER
I20260811 22:24:22.510496 1569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8789570fb8de4cb1956e3c8a169aa4f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36521 } }
I20260811 22:24:22.510749 1569 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:22.510921 1569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:22.511129 1569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:22.513787 1569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8789570fb8de4cb1956e3c8a169aa4f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36521 } }
I20260811 22:24:22.514261 1569 leader_election.cc:302] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8789570fb8de4cb1956e3c8a169aa4f3; no voters:
I20260811 22:24:22.514858 1569 leader_election.cc:288] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:22.514998 1583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:22.515697 1583 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3 [term 1 LEADER]: Becoming Leader. State: Replica: 8789570fb8de4cb1956e3c8a169aa4f3, State: Running, Role: LEADER
I20260811 22:24:22.516409 1583 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8789570fb8de4cb1956e3c8a169aa4f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36521 } }
I20260811 22:24:22.517730 1569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:22.519238 1584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8789570fb8de4cb1956e3c8a169aa4f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8789570fb8de4cb1956e3c8a169aa4f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36521 } } }
I20260811 22:24:22.519891 1584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:22.520877 1585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8789570fb8de4cb1956e3c8a169aa4f3. Latest consensus state: current_term: 1 leader_uuid: "8789570fb8de4cb1956e3c8a169aa4f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8789570fb8de4cb1956e3c8a169aa4f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36521 } } }
I20260811 22:24:22.521476 1585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:22.522882 1592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:22.529147 1592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:22.538949 1592 catalog_manager.cc:1347] Generated new cluster ID: 72d2efd720f7414a9551cae7dee8fdf1
I20260811 22:24:22.539081 1592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:22.558275 1592 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:22.558780 1592 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:22.572806 1592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8789570fb8de4cb1956e3c8a169aa4f3: Generated new TSK 0
I20260811 22:24:22.573700 1592 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:22.580346 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 1456
W20260811 22:24:22.581607 1564 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.48:39183
W20260811 22:24:22.582365 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 1456: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:24:22.582554 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 1456
2026-08-11T22:24:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2420 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:22 dist-test-slave-jq4t krb5kdc[1617](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:22 dist-test-slave-jq4t krb5kdc[1617](info): set up 2 sockets
Aug 11 22:24:22 dist-test-slave-jq4t krb5kdc[1617](info): commencing operation
krb5kdc: starting...
W20260811 22:24:24.029402 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.396s user 0.000s sys 0.068s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:24 dist-test-slave-jq4t krb5kdc[1617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:24:24.444123 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44785
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:24.817022 1646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:24.817344 1646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:24.817477 1646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:24.825351 1646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:24.825487 1646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:24.825569 1646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:24.825688 1646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:24.825749 1646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:24.836535 1646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44785
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:24.838181 1646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:24.839643 1646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:24.854350 1654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:24.854301 1655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:24.854386 1658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:24.855352 1646 server_base.cc:1034] running on GCE node
I20260811 22:24:24.856696 1646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:24.858916 1646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:24.860248 1646 hybrid_clock.cc:648] HybridClock initialized: now 1786487064860145 us; error 84 us; skew 500 ppm
I20260811 22:24:24.860826 1646 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:24 dist-test-slave-jq4t krb5kdc[1617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487064, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:24.865260 1646 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:24:24.867529 1646 webserver.cc:460] Webserver started at http://127.0.0.1:33135/ using document root <none> and password file <none>
I20260811 22:24:24.868351 1646 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:24.868566 1646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:24.868889 1646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:24.872237 1646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "65fa85b2a1db44fdb7fba5d9483cc8ad"
format_stamp: "Formatted at 2026-08-11 22:24:24 on dist-test-slave-jq4t"
server_key: "afa28f96b719dc6b316d6e2087c7b6b6"
server_key_iv: "d62ad0340c73df897f675a369cfb8169"
server_key_version: "encryptionkey@0"
I20260811 22:24:24.873117 1646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "65fa85b2a1db44fdb7fba5d9483cc8ad"
format_stamp: "Formatted at 2026-08-11 22:24:24 on dist-test-slave-jq4t"
server_key: "afa28f96b719dc6b316d6e2087c7b6b6"
server_key_iv: "d62ad0340c73df897f675a369cfb8169"
server_key_version: "encryptionkey@0"
I20260811 22:24:24.878978 1646 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:24:24.883224 1666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:24.885125 1646 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 22:24:24.885303 1646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "65fa85b2a1db44fdb7fba5d9483cc8ad"
format_stamp: "Formatted at 2026-08-11 22:24:24 on dist-test-slave-jq4t"
server_key: "afa28f96b719dc6b316d6e2087c7b6b6"
server_key_iv: "d62ad0340c73df897f675a369cfb8169"
server_key_version: "encryptionkey@0"
I20260811 22:24:24.885547 1646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:24.912971 1646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:24.918133 1646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:24.918457 1646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:24.938977 1646 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44785
I20260811 22:24:24.939019 1762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44785 every 8 connection(s)
I20260811 22:24:24.940424 1646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:24.945005 1763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:24.947070 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 1646
I20260811 22:24:24.947284 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:24.947755 30612 external_mini_cluster.cc:1428] Setting key 8588a5bc9d33f6411b47440aaded9c9c
I20260811 22:24:24.957878 1763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad: Bootstrap starting.
I20260811 22:24:24.963107 1763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:24.965052 1763 log.cc:824] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:24:24 dist-test-slave-jq4t krb5kdc[1617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:24:24.968957 1763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad: No bootstrap required, opened a new log
I20260811 22:24:24.975607 1766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:24.949698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:39337 (local address 127.0.0.1:44785)
0811 22:24:24.950687 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:24.950700 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:24.950748 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:24:24.952428 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:24.952450 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:24.952913 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:24.953236 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:24.955273 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:24.957150 (+ 1877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:24.959048 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:24.960238 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:24.960458 (+ 220us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:24.967077 (+ 6619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:24.967087 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:24.967105 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:24.967148 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:24.970710 (+ 3562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:24.971959 (+ 1249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:24.971966 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:24.971973 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:24.972156 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:24.972707 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:24.972710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:24.972713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:24.973255 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:24.973516 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:24.974121 (+ 605us) server_negotiation.cc:299] Negotiation successful
0811 22:24:24.974478 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":765,"thread_start_us":490,"threads_started":1}
I20260811 22:24:24.979283 1763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65fa85b2a1db44fdb7fba5d9483cc8ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44785 } }
I20260811 22:24:24.979755 1763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:24.979875 1763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65fa85b2a1db44fdb7fba5d9483cc8ad, State: Initialized, Role: FOLLOWER
I20260811 22:24:24.980419 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:24.980551 1763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65fa85b2a1db44fdb7fba5d9483cc8ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44785 } }
I20260811 22:24:24.980819 1763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:24.980967 1763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:24.981153 1763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:24.984066 1763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65fa85b2a1db44fdb7fba5d9483cc8ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44785 } }
I20260811 22:24:24.984714 1763 leader_election.cc:302] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65fa85b2a1db44fdb7fba5d9483cc8ad; no voters:
I20260811 22:24:24.985339 1763 leader_election.cc:288] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:24.985495 1775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:24.987488 1775 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad [term 1 LEADER]: Becoming Leader. State: Replica: 65fa85b2a1db44fdb7fba5d9483cc8ad, State: Running, Role: LEADER
I20260811 22:24:24.988175 1775 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65fa85b2a1db44fdb7fba5d9483cc8ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44785 } }
I20260811 22:24:24.988791 1763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:24.991292 1776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65fa85b2a1db44fdb7fba5d9483cc8ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65fa85b2a1db44fdb7fba5d9483cc8ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44785 } } }
I20260811 22:24:24.991786 1776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:24.993059 1778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65fa85b2a1db44fdb7fba5d9483cc8ad. Latest consensus state: current_term: 1 leader_uuid: "65fa85b2a1db44fdb7fba5d9483cc8ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65fa85b2a1db44fdb7fba5d9483cc8ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44785 } } }
I20260811 22:24:24.993433 1778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:24.994168 1788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:25.000164 1788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:25.013214 1766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:24.987313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:49491 (local address 127.0.0.1:44785)
0811 22:24:24.987511 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:24.987519 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:24.987550 (+ 31us) server_negotiation.cc:407] Connection header received
0811 22:24:24.987854 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:24.987860 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:24.987945 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:24.988076 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:24.993105 (+ 5029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:24.994498 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:24.997173 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:24.998355 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:24.998547 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:25.003777 (+ 5230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:25.003789 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:25.003801 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:25.003866 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:25.008558 (+ 4692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:25.009774 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:25.009780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:25.009784 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:25.009942 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:25.010597 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:25.010603 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:25.010608 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:25.011106 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:25.012398 (+ 1292us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:25.012621 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 22:24:25.012819 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:24:25.018890 1788 catalog_manager.cc:1347] Generated new cluster ID: 03c477a4f3874ef88bd1c6031815b735
I20260811 22:24:25.019023 1788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:25.030931 1788 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:25.032990 1788 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:25.040863 1788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65fa85b2a1db44fdb7fba5d9483cc8ad: Generated new TSK 0
I20260811 22:24:25.041733 1788 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:25.052337 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 1646
I20260811 22:24:25.053860 1756 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:24:25.153082 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 1646
W20260811 22:24:25.183243 1614 connection.cc:535] client connection to 127.0.0.1:44785 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44785): Connection reset by peer (error 104)
2026-08-11T22:24:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2569 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:25 dist-test-slave-jq4t krb5kdc[1824](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:25 dist-test-slave-jq4t krb5kdc[1824](info): set up 2 sockets
Aug 11 22:24:25 dist-test-slave-jq4t krb5kdc[1824](info): commencing operation
krb5kdc: starting...
W20260811 22:24:26.591956 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.388s user 0.000s sys 0.067s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:26 dist-test-slave-jq4t krb5kdc[1824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:24:27.049116 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33073
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:27.483322 1842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:27.483660 1842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:27.483763 1842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:27.491729 1842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:27.491885 1842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:27.491943 1842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:27.492018 1842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:27.492066 1842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:27.503185 1842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33073
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:27.505337 1842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:27.507190 1842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:27.521672 1855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:27.521782 1860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:27.521806 1856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:27.522881 1842 server_base.cc:1034] running on GCE node
I20260811 22:24:27.523722 1842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:27.525417 1842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:27.526815 1842 hybrid_clock.cc:648] HybridClock initialized: now 1786487067526723 us; error 31 us; skew 500 ppm
I20260811 22:24:27.527457 1842 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:27 dist-test-slave-jq4t krb5kdc[1824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487067, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:27.534271 1842 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:24:27.536079 1842 webserver.cc:460] Webserver started at http://127.0.0.1:36043/ using document root <none> and password file <none>
I20260811 22:24:27.536918 1842 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:27.537074 1842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:27.537529 1842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:27.540755 1842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "678e8957decd40128a4dc4aae35ef4ca"
format_stamp: "Formatted at 2026-08-11 22:24:27 on dist-test-slave-jq4t"
server_key: "cbec627c2d1cdda415f0036c3957305c"
server_key_iv: "c3a8951bfd523f3d514727828ea7d26a"
server_key_version: "encryptionkey@0"
I20260811 22:24:27.541574 1842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "678e8957decd40128a4dc4aae35ef4ca"
format_stamp: "Formatted at 2026-08-11 22:24:27 on dist-test-slave-jq4t"
server_key: "cbec627c2d1cdda415f0036c3957305c"
server_key_iv: "c3a8951bfd523f3d514727828ea7d26a"
server_key_version: "encryptionkey@0"
I20260811 22:24:27.547125 1842 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 22:24:27.550873 1869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:27.552325 1842 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:24:27.552604 1842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "678e8957decd40128a4dc4aae35ef4ca"
format_stamp: "Formatted at 2026-08-11 22:24:27 on dist-test-slave-jq4t"
server_key: "cbec627c2d1cdda415f0036c3957305c"
server_key_iv: "c3a8951bfd523f3d514727828ea7d26a"
server_key_version: "encryptionkey@0"
I20260811 22:24:27.552831 1842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:27.566670 1842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:27.573204 1842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:27.573529 1842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:27.595997 1842 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33073
I20260811 22:24:27.596019 1945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33073 every 8 connection(s)
I20260811 22:24:27.597643 1842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:27.598706 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 1842
I20260811 22:24:27.598889 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:27.599306 30612 external_mini_cluster.cc:1428] Setting key e1c648560736f78e3fda2946137d1a76
I20260811 22:24:27.602306 1948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:27.614444 1948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca: Bootstrap starting.
I20260811 22:24:27.614784 1959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:27.601276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:39443 (local address 127.0.0.1:33073)
0811 22:24:27.602110 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:27.602122 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:27.602165 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:24:27.604719 (+ 2554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:27.604745 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:27.605172 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:27.605610 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:27.606193 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:27.607630 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:27.609101 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:27.609783 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:27.609998 (+ 215us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:27.611869 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:27.611880 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:27.611903 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:27.612859 (+ 956us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:27.612951 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:27.612982 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:27.613182 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:27.613504 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 22:24:27.613695 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":364,"threads_started":1}
I20260811 22:24:27.618105 1948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:27.619364 1948 log.cc:824] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:27.619451 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:27.622303 1948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca: No bootstrap required, opened a new log
I20260811 22:24:27.631983 1959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:27.624352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:60241 (local address 127.0.0.1:33073)
0811 22:24:27.624610 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:27.624618 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:27.624953 (+ 335us) server_negotiation.cc:407] Connection header received
0811 22:24:27.625160 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:27.625164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:27.625240 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:27.625333 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:27.625927 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:27.626730 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:27.628726 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:27.629229 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:27.629344 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:27.630750 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:27.630755 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:27.630758 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:27.630779 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:27.630845 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:27.630861 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:27.630956 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:27.631430 (+ 474us) server_negotiation.cc:299] Negotiation successful
0811 22:24:27.631541 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 22:24:27.632705 1948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678e8957decd40128a4dc4aae35ef4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33073 } }
I20260811 22:24:27.633190 1948 raft_consensus.cc:374] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:27.633330 1948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 678e8957decd40128a4dc4aae35ef4ca, State: Initialized, Role: FOLLOWER
I20260811 22:24:27.633970 1948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678e8957decd40128a4dc4aae35ef4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33073 } }
I20260811 22:24:27.634248 1948 raft_consensus.cc:388] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:27.634418 1948 raft_consensus.cc:482] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:27.634614 1948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:27.636907 1948 raft_consensus.cc:504] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678e8957decd40128a4dc4aae35ef4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33073 } }
I20260811 22:24:27.637384 1948 leader_election.cc:302] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 678e8957decd40128a4dc4aae35ef4ca; no voters:
I20260811 22:24:27.637950 1948 leader_election.cc:288] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:27.638123 1969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:27.638895 1969 raft_consensus.cc:686] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca [term 1 LEADER]: Becoming Leader. State: Replica: 678e8957decd40128a4dc4aae35ef4ca, State: Running, Role: LEADER
I20260811 22:24:27.639536 1969 consensus_queue.cc:237] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678e8957decd40128a4dc4aae35ef4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33073 } }
I20260811 22:24:27.640352 1948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:27.646265 1972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 678e8957decd40128a4dc4aae35ef4ca. Latest consensus state: current_term: 1 leader_uuid: "678e8957decd40128a4dc4aae35ef4ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678e8957decd40128a4dc4aae35ef4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33073 } } }
I20260811 22:24:27.646710 1972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:27.647317 1971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "678e8957decd40128a4dc4aae35ef4ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678e8957decd40128a4dc4aae35ef4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33073 } } }
I20260811 22:24:27.647610 1971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:27.647637 1989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:27.656199 1989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:27.666392 1989 catalog_manager.cc:1347] Generated new cluster ID: 724e4ae92a4f4c4aa49dd67b489de440
I20260811 22:24:27.666566 1989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:27.678241 1989 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:27.679854 1989 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:27.690706 1989 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 678e8957decd40128a4dc4aae35ef4ca: Generated new TSK 0
I20260811 22:24:27.691819 1989 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:27.709936 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 1842
W20260811 22:24:27.711215 1940 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.48:39443
W20260811 22:24:27.712005 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 1842: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:24:27.712162 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 1842
W20260811 22:24:27.743755 1822 connection.cc:535] client connection to 127.0.0.1:33073 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33073): Connection reset by peer (error 104)
2026-08-11T22:24:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2560 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:27 dist-test-slave-jq4t krb5kdc[2008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:27 dist-test-slave-jq4t krb5kdc[2008](info): set up 2 sockets
Aug 11 22:24:27 dist-test-slave-jq4t krb5kdc[2008](info): commencing operation
krb5kdc: starting...
W20260811 22:24:29.176270 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.412s user 0.013s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:29 dist-test-slave-jq4t krb5kdc[2008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:24:29.588699 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46533
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:29.974397 2031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:29.974722 2031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:29.974843 2031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:29.982491 2031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:29.982640 2031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:29.982690 2031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:29.982749 2031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:29.982796 2031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:29.993415 2031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46533
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:29.994843 2031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:29.996228 2031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:30.009877 2040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:30.009917 2045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:30.009877 2041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:30.010839 2031 server_base.cc:1034] running on GCE node
I20260811 22:24:30.011727 2031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:30.013491 2031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:30.014806 2031 hybrid_clock.cc:648] HybridClock initialized: now 1786487070014751 us; error 65 us; skew 500 ppm
I20260811 22:24:30.015237 2031 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:30 dist-test-slave-jq4t krb5kdc[2008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487070, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:30.018850 2031 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:24:30.021170 2031 webserver.cc:460] Webserver started at http://127.0.0.1:41889/ using document root <none> and password file <none>
I20260811 22:24:30.021956 2031 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:30.022141 2031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:30.022733 2031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:30.026515 2031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "587173f710db42e1b8b2a9ca454d0db5"
format_stamp: "Formatted at 2026-08-11 22:24:30 on dist-test-slave-jq4t"
server_key: "b3615a3743120c0925369b26abc3105b"
server_key_iv: "a2affb6e6800e8ca19983f5d6ad70843"
server_key_version: "encryptionkey@0"
I20260811 22:24:30.027426 2031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "587173f710db42e1b8b2a9ca454d0db5"
format_stamp: "Formatted at 2026-08-11 22:24:30 on dist-test-slave-jq4t"
server_key: "b3615a3743120c0925369b26abc3105b"
server_key_iv: "a2affb6e6800e8ca19983f5d6ad70843"
server_key_version: "encryptionkey@0"
I20260811 22:24:30.033780 2031 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 22:24:30.039330 2055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:30.041397 2031 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 22:24:30.041656 2031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "587173f710db42e1b8b2a9ca454d0db5"
format_stamp: "Formatted at 2026-08-11 22:24:30 on dist-test-slave-jq4t"
server_key: "b3615a3743120c0925369b26abc3105b"
server_key_iv: "a2affb6e6800e8ca19983f5d6ad70843"
server_key_version: "encryptionkey@0"
I20260811 22:24:30.041939 2031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:30.048673 2031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:30.053512 2031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:30.053866 2031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:30.078208 2031 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46533
I20260811 22:24:30.078347 2139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46533 every 8 connection(s)
I20260811 22:24:30.079774 2031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:30.084790 2148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:30.090276 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 2031
I20260811 22:24:30.090534 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:30.091270 30612 external_mini_cluster.cc:1428] Setting key 994b701d693826230f1cb10c81e93a71
I20260811 22:24:30.100301 2148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5: Bootstrap starting.
I20260811 22:24:30.102439 2152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:30.094038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:50871 (local address 127.0.0.1:46533)
0811 22:24:30.095328 (+ 1290us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:30.095344 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:30.095416 (+ 72us) server_negotiation.cc:407] Connection header received
0811 22:24:30.097968 (+ 2552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:30.097998 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:30.098436 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:30.098848 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:30.100322 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:30.100329 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:30.100343 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:30.100361 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:30.100439 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:30.100538 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:30.100826 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:30.100947 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 22:24:30.101215 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":973,"thread_start_us":529,"threads_started":1}
I20260811 22:24:30.105351 2148 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:30.106653 2148 log.cc:824] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:30.107267 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:30.110025 2148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5: No bootstrap required, opened a new log
I20260811 22:24:30.115545 2152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:30.112610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:56007 (local address 127.0.0.1:46533)
0811 22:24:30.112798 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:30.112803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:30.113154 (+ 351us) server_negotiation.cc:407] Connection header received
0811 22:24:30.113397 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:30.113411 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:30.113468 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:30.113613 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:30.114717 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:30.114724 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:30.114729 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:30.114754 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:30.114806 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:30.114827 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:30.114981 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:30.115135 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 22:24:30.115225 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:24:30.119403 2148 raft_consensus.cc:348] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587173f710db42e1b8b2a9ca454d0db5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46533 } }
I20260811 22:24:30.119855 2148 raft_consensus.cc:374] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:30.120007 2148 raft_consensus.cc:729] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 587173f710db42e1b8b2a9ca454d0db5, State: Initialized, Role: FOLLOWER
I20260811 22:24:30.120713 2148 consensus_queue.cc:260] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587173f710db42e1b8b2a9ca454d0db5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46533 } }
I20260811 22:24:30.121006 2148 raft_consensus.cc:388] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:30.121253 2148 raft_consensus.cc:482] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:30.121521 2148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:30.124205 2148 raft_consensus.cc:504] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587173f710db42e1b8b2a9ca454d0db5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46533 } }
I20260811 22:24:30.124893 2148 leader_election.cc:302] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 587173f710db42e1b8b2a9ca454d0db5; no voters:
I20260811 22:24:30.125658 2148 leader_election.cc:288] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:30.125813 2170 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:30.126878 2170 raft_consensus.cc:686] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5 [term 1 LEADER]: Becoming Leader. State: Replica: 587173f710db42e1b8b2a9ca454d0db5, State: Running, Role: LEADER
I20260811 22:24:30.127630 2170 consensus_queue.cc:237] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587173f710db42e1b8b2a9ca454d0db5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46533 } }
I20260811 22:24:30.128578 2148 sys_catalog.cc:564] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:30.131110 2172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 587173f710db42e1b8b2a9ca454d0db5. Latest consensus state: current_term: 1 leader_uuid: "587173f710db42e1b8b2a9ca454d0db5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587173f710db42e1b8b2a9ca454d0db5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46533 } } }
I20260811 22:24:30.131596 2172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:30.130816 2171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "587173f710db42e1b8b2a9ca454d0db5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587173f710db42e1b8b2a9ca454d0db5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46533 } } }
I20260811 22:24:30.132288 2171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:30.133935 2180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:30.141016 2180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:30.151180 2180 catalog_manager.cc:1347] Generated new cluster ID: a5133fcde7ee4b50bd4be7b94240f245
I20260811 22:24:30.151368 2180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:30.165789 2180 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:30.166290 2180 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:30.173499 2180 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 587173f710db42e1b8b2a9ca454d0db5: Generated new TSK 0
I20260811 22:24:30.174378 2180 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:30.183732 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 2031
W20260811 22:24:30.185164 2129 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.48:50871
W20260811 22:24:30.186115 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 2031: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:24:30.186347 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 2031
2026-08-11T22:24:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2477 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:30 dist-test-slave-jq4t krb5kdc[2210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:30 dist-test-slave-jq4t krb5kdc[2210](info): set up 2 sockets
Aug 11 22:24:30 dist-test-slave-jq4t krb5kdc[2210](info): commencing operation
krb5kdc: starting...
W20260811 22:24:31.651921 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.410s user 0.000s sys 0.069s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:31 dist-test-slave-jq4t krb5kdc[2210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:24:32.060443 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:45581
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:39687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:45581
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:32.437928 2233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:32.438338 2233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:32.438514 2233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:32.446170 2233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:32.446358 2233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:32.446472 2233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:32.446588 2233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:32.446693 2233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:32.456990 2233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:39687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:45581
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:45581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:32.458438 2233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:32.459801 2233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:32.473049 2246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:32.472999 2252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:32.473063 2247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:32.474582 2233 server_base.cc:1034] running on GCE node
I20260811 22:24:32.475914 2233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:32.477686 2233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:32.479030 2233 hybrid_clock.cc:648] HybridClock initialized: now 1786487072478943 us; error 43 us; skew 500 ppm
I20260811 22:24:32.479897 2233 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:32 dist-test-slave-jq4t krb5kdc[2210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487072, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:32.484153 2233 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:24:32.486171 2233 webserver.cc:460] Webserver started at http://127.29.229.62:43601/ using document root <none> and password file <none>
I20260811 22:24:32.486994 2233 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:32.487128 2233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:32.487478 2233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:32.491873 2233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "11f09c352fc445eb9c2cc65115f50d66"
format_stamp: "Formatted at 2026-08-11 22:24:32 on dist-test-slave-jq4t"
server_key: "71118c3e142f79b4e4fa8c7ef04b6ff6"
server_key_iv: "2ff706eee12745b1b3b544eb15c7a399"
server_key_version: "encryptionkey@0"
I20260811 22:24:32.492875 2233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "11f09c352fc445eb9c2cc65115f50d66"
format_stamp: "Formatted at 2026-08-11 22:24:32 on dist-test-slave-jq4t"
server_key: "71118c3e142f79b4e4fa8c7ef04b6ff6"
server_key_iv: "2ff706eee12745b1b3b544eb15c7a399"
server_key_version: "encryptionkey@0"
I20260811 22:24:32.498564 2233 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:24:32.502499 2267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:32.504268 2233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:24:32.504446 2233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "11f09c352fc445eb9c2cc65115f50d66"
format_stamp: "Formatted at 2026-08-11 22:24:32 on dist-test-slave-jq4t"
server_key: "71118c3e142f79b4e4fa8c7ef04b6ff6"
server_key_iv: "2ff706eee12745b1b3b544eb15c7a399"
server_key_version: "encryptionkey@0"
I20260811 22:24:32.504737 2233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:32.519587 2233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:32.525624 2233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:32.525915 2233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:32.552428 2233 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:45581
I20260811 22:24:32.552461 2361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:45581 every 8 connection(s)
I20260811 22:24:32.554141 2233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:32.558136 2362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:32.562415 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 2233
I20260811 22:24:32.562640 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:32.563136 30612 external_mini_cluster.cc:1428] Setting key 5b3ba6143e05539eced0a654da6145dc
I20260811 22:24:32.570221 2362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66: Bootstrap starting.
I20260811 22:24:32.574435 2362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:32.575753 2362 log.cc:824] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:32.578846 2362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66: No bootstrap required, opened a new log
Aug 11 22:24:32 dist-test-slave-jq4t krb5kdc[2210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:24:32.589561 2362 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f09c352fc445eb9c2cc65115f50d66" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45581 } }
I20260811 22:24:32.590101 2362 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:32.590224 2362 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f09c352fc445eb9c2cc65115f50d66, State: Initialized, Role: FOLLOWER
I20260811 22:24:32.590917 2362 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f09c352fc445eb9c2cc65115f50d66" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45581 } }
I20260811 22:24:32.591220 2362 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:32.591418 2362 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:32.591624 2362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:32.595268 2362 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f09c352fc445eb9c2cc65115f50d66" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45581 } }
I20260811 22:24:32.595575 2372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:32.565237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34901 (local address 127.29.229.62:45581)
0811 22:24:32.566084 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:32.566098 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:32.566142 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:24:32.568901 (+ 2759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:32.568931 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:32.569388 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:32.569758 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:32.571808 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:32.573364 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:32.575311 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:32.576119 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:32.576325 (+ 206us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:32.585306 (+ 8981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:32.585321 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:32.585345 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:32.585415 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:32.589895 (+ 4480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:32.591195 (+ 1300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:32.591210 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:32.591220 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:32.591458 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:32.592090 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:32.592095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:32.592098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:32.592939 (+ 841us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:32.593306 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:32.593773 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 22:24:32.594164 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":385,"threads_started":1}
I20260811 22:24:32.595919 2362 leader_election.cc:302] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11f09c352fc445eb9c2cc65115f50d66; no voters:
I20260811 22:24:32.596705 2362 leader_election.cc:288] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:32.596834 2375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:32.598241 2375 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66 [term 1 LEADER]: Becoming Leader. State: Replica: 11f09c352fc445eb9c2cc65115f50d66, State: Running, Role: LEADER
I20260811 22:24:32.598881 2375 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f09c352fc445eb9c2cc65115f50d66" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45581 } }
I20260811 22:24:32.599738 2362 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:32.601046 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:32.601755 2378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11f09c352fc445eb9c2cc65115f50d66. Latest consensus state: current_term: 1 leader_uuid: "11f09c352fc445eb9c2cc65115f50d66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f09c352fc445eb9c2cc65115f50d66" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45581 } } }
I20260811 22:24:32.602177 2378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:32.609083 2381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:32.609504 2376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11f09c352fc445eb9c2cc65115f50d66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f09c352fc445eb9c2cc65115f50d66" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 45581 } } }
I20260811 22:24:32.610105 2376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:32.614984 2381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:32.634235 2381 catalog_manager.cc:1347] Generated new cluster ID: a724311ba894492b88161ea255c37dd3
I20260811 22:24:32.634370 2381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:32.647815 2372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:32.612374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40181 (local address 127.29.229.62:45581)
0811 22:24:32.612657 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:32.612669 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:32.620528 (+ 7859us) server_negotiation.cc:407] Connection header received
0811 22:24:32.621202 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:32.621212 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:32.621333 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:32.621443 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:32.626113 (+ 4670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:32.627352 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:32.629374 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:32.630064 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:32.630176 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:32.636322 (+ 6146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:32.636335 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:32.636342 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:32.636405 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:32.641544 (+ 5139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:32.643101 (+ 1557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:32.643110 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:32.643116 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:32.643345 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:32.644087 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:32.644095 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:32.644100 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:32.644641 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:32.645006 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:32.646761 (+ 1755us) server_negotiation.cc:299] Negotiation successful
0811 22:24:32.647217 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:24:32.653412 2381 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:32.655913 2381 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:32.666844 2381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11f09c352fc445eb9c2cc65115f50d66: Generated new TSK 0
I20260811 22:24:32.667994 2381 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:32.679108 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 2233
I20260811 22:24:32.681243 2357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:24:32.779898 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 2233
W20260811 22:24:32.805716 2206 connection.cc:535] client connection to 127.29.229.62:45581 recv error: Network error: failed to read from TLS socket (remote: 127.29.229.62:45581): Connection reset by peer (error 104)
2026-08-11T22:24:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2583 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:32 dist-test-slave-jq4t krb5kdc[2423](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:32 dist-test-slave-jq4t krb5kdc[2423](info): set up 2 sockets
Aug 11 22:24:32 dist-test-slave-jq4t krb5kdc[2423](info): commencing operation
krb5kdc: starting...
W20260811 22:24:34.214701 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.390s user 0.003s sys 0.063s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:34 dist-test-slave-jq4t krb5kdc[2423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:24:34.652241 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:44947
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:44947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:35.046808 2448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:35.047271 2448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:35.047509 2448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:35.055686 2448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:35.055836 2448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:35.055940 2448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:35.056039 2448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:35.056084 2448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:35.068457 2448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:44947
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:44947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:35.070420 2448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:35.072296 2448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:35.088621 2463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:35.088680 2459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:35.088557 2460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:35.089516 2448 server_base.cc:1034] running on GCE node
I20260811 22:24:35.090448 2448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:35.092162 2448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:35.093451 2448 hybrid_clock.cc:648] HybridClock initialized: now 1786487075093376 us; error 67 us; skew 500 ppm
I20260811 22:24:35.093896 2448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:35 dist-test-slave-jq4t krb5kdc[2423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487075, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:35.097604 2448 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:24:35.100111 2448 webserver.cc:460] Webserver started at http://127.29.229.62:36745/ using document root <none> and password file <none>
I20260811 22:24:35.101353 2448 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:35.101583 2448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:35.102027 2448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:35.106280 2448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "ec7ee6545ce44992872c28163408ce73"
format_stamp: "Formatted at 2026-08-11 22:24:35 on dist-test-slave-jq4t"
server_key: "93346a0b989c23f97fac31ddfa40ca53"
server_key_iv: "9656c3fc612b9f48c0f5d44caa21b71f"
server_key_version: "encryptionkey@0"
I20260811 22:24:35.107237 2448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "ec7ee6545ce44992872c28163408ce73"
format_stamp: "Formatted at 2026-08-11 22:24:35 on dist-test-slave-jq4t"
server_key: "93346a0b989c23f97fac31ddfa40ca53"
server_key_iv: "9656c3fc612b9f48c0f5d44caa21b71f"
server_key_version: "encryptionkey@0"
I20260811 22:24:35.114523 2448 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.003s
I20260811 22:24:35.119046 2483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:35.120715 2448 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:24:35.120932 2448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "ec7ee6545ce44992872c28163408ce73"
format_stamp: "Formatted at 2026-08-11 22:24:35 on dist-test-slave-jq4t"
server_key: "93346a0b989c23f97fac31ddfa40ca53"
server_key_iv: "9656c3fc612b9f48c0f5d44caa21b71f"
server_key_version: "encryptionkey@0"
I20260811 22:24:35.121165 2448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:35.144220 2448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:35.149048 2448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:35.149391 2448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:35.172585 2448 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:44947
I20260811 22:24:35.172598 2563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:44947 every 8 connection(s)
I20260811 22:24:35.174250 2448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:35.175802 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 2448
I20260811 22:24:35.175997 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:35.176626 30612 external_mini_cluster.cc:1428] Setting key b91e4021b2b609d355861bf7d06ae079
I20260811 22:24:35.179364 2564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:35.193419 2568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:35.179205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51567 (local address 127.29.229.62:44947)
0811 22:24:35.180336 (+ 1131us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:35.180349 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:35.180403 (+ 54us) server_negotiation.cc:407] Connection header received
0811 22:24:35.182185 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:35.182208 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:35.182617 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:35.182987 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:35.183998 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:35.185591 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:35.188297 (+ 2706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:35.189139 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:35.189301 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:35.190818 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:35.190827 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:35.190842 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:35.191432 (+ 590us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:35.191534 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:35.191564 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:35.191692 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:35.192075 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 22:24:35.192296 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":894,"thread_start_us":630,"threads_started":1}
I20260811 22:24:35.195384 2564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73: Bootstrap starting.
I20260811 22:24:35.198839 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:35.199223 2564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:35.200297 2564 log.cc:824] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:35.203979 2564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73: No bootstrap required, opened a new log
I20260811 22:24:35.212831 2568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:35.203981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60979 (local address 127.29.229.62:44947)
0811 22:24:35.204226 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:35.204233 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:35.204972 (+ 739us) server_negotiation.cc:407] Connection header received
0811 22:24:35.205127 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:35.205132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:35.205231 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:35.205363 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:35.206611 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:35.207554 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:35.209644 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:35.210305 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:35.210418 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:35.211861 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:35.211866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:35.211869 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:35.211892 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:35.211932 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:35.211956 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:35.212060 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:35.212332 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 22:24:35.212436 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260811 22:24:35.215452 2564 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec7ee6545ce44992872c28163408ce73" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44947 } }
I20260811 22:24:35.215885 2564 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:35.216003 2564 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec7ee6545ce44992872c28163408ce73, State: Initialized, Role: FOLLOWER
I20260811 22:24:35.216712 2564 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec7ee6545ce44992872c28163408ce73" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44947 } }
I20260811 22:24:35.216966 2564 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:35.217125 2564 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:35.217327 2564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:35.219867 2564 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec7ee6545ce44992872c28163408ce73" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44947 } }
I20260811 22:24:35.220428 2564 leader_election.cc:302] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec7ee6545ce44992872c28163408ce73; no voters:
I20260811 22:24:35.221232 2564 leader_election.cc:288] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:35.221556 2583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:35.222440 2583 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73 [term 1 LEADER]: Becoming Leader. State: Replica: ec7ee6545ce44992872c28163408ce73, State: Running, Role: LEADER
I20260811 22:24:35.223184 2583 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec7ee6545ce44992872c28163408ce73" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44947 } }
I20260811 22:24:35.223719 2564 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:35.226814 2586 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec7ee6545ce44992872c28163408ce73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec7ee6545ce44992872c28163408ce73" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44947 } } }
I20260811 22:24:35.227384 2586 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:35.228011 2584 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec7ee6545ce44992872c28163408ce73. Latest consensus state: current_term: 1 leader_uuid: "ec7ee6545ce44992872c28163408ce73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec7ee6545ce44992872c28163408ce73" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44947 } } }
I20260811 22:24:35.228368 2584 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:35.229744 2597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:35.235570 2597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:35.246908 2597 catalog_manager.cc:1347] Generated new cluster ID: e09c612a5cd441209c57770ff3ef0772
I20260811 22:24:35.247051 2597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:35.279167 2597 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:35.280722 2597 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:35.288192 2597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec7ee6545ce44992872c28163408ce73: Generated new TSK 0
I20260811 22:24:35.289162 2597 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:35.297803 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 2448
W20260811 22:24:35.299297 2557 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51567
W20260811 22:24:35.300263 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 2448: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:24:35.300403 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 2448
W20260811 22:24:35.325997 2421 connection.cc:535] client connection to 127.29.229.62:44947 recv error: Network error: failed to read from TLS socket (remote: 127.29.229.62:44947): Connection reset by peer (error 104)
2026-08-11T22:24:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2521 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:35 dist-test-slave-jq4t krb5kdc[2623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:35 dist-test-slave-jq4t krb5kdc[2623](info): set up 2 sockets
Aug 11 22:24:35 dist-test-slave-jq4t krb5kdc[2623](info): commencing operation
krb5kdc: starting...
W20260811 22:24:36.748220 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.402s user 0.000s sys 0.068s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:37 dist-test-slave-jq4t krb5kdc[2623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:24:37.168668 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:40579
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:40579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:37.573128 2647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:37.573439 2647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:37.573606 2647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:37.581089 2647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:37.581205 2647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:37.581252 2647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:37.581347 2647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:37.581435 2647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:37.591511 2647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:40579
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:40579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:37.592985 2647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:37.594297 2647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:37.606559 2659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:37.606639 2660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:37.606662 2665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:37.607841 2647 server_base.cc:1034] running on GCE node
I20260811 22:24:37.608790 2647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:37.610360 2647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:37.611608 2647 hybrid_clock.cc:648] HybridClock initialized: now 1786487077611565 us; error 33 us; skew 500 ppm
I20260811 22:24:37.612038 2647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:37 dist-test-slave-jq4t krb5kdc[2623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487077, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:37.618999 2647 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:24:37.620784 2647 webserver.cc:460] Webserver started at http://127.29.229.62:37565/ using document root <none> and password file <none>
I20260811 22:24:37.621539 2647 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:37.621701 2647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:37.622043 2647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:37.625409 2647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "01249faff3a443f8bf06af81e28982e6"
format_stamp: "Formatted at 2026-08-11 22:24:37 on dist-test-slave-jq4t"
server_key: "065ae9901f47759b9a53a5e6ce297c5a"
server_key_iv: "76fd62daf4867a366a63ef9e4ab2a8d6"
server_key_version: "encryptionkey@0"
I20260811 22:24:37.626220 2647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "01249faff3a443f8bf06af81e28982e6"
format_stamp: "Formatted at 2026-08-11 22:24:37 on dist-test-slave-jq4t"
server_key: "065ae9901f47759b9a53a5e6ce297c5a"
server_key_iv: "76fd62daf4867a366a63ef9e4ab2a8d6"
server_key_version: "encryptionkey@0"
I20260811 22:24:37.632021 2647 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 22:24:37.636003 2676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:37.637459 2647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:24:37.637681 2647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "01249faff3a443f8bf06af81e28982e6"
format_stamp: "Formatted at 2026-08-11 22:24:37 on dist-test-slave-jq4t"
server_key: "065ae9901f47759b9a53a5e6ce297c5a"
server_key_iv: "76fd62daf4867a366a63ef9e4ab2a8d6"
server_key_version: "encryptionkey@0"
I20260811 22:24:37.637923 2647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:37.644382 2647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:37.649075 2647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:37.649430 2647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:37.671521 2647 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:40579
I20260811 22:24:37.671561 2770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:40579 every 8 connection(s)
I20260811 22:24:37.673066 2647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:37.673753 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 2647
I20260811 22:24:37.673978 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:37.674606 30612 external_mini_cluster.cc:1428] Setting key 2c70c3ba356d5fb1b0798fcce4035670
I20260811 22:24:37.678059 2771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:37.684381 2775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:37.676699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48329 (local address 127.29.229.62:40579)
0811 22:24:37.677570 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:37.677587 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:37.677642 (+ 55us) server_negotiation.cc:407] Connection header received
0811 22:24:37.680024 (+ 2382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:37.680053 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:37.680533 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:37.680997 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:37.682304 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:37.682314 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:37.682331 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:37.682359 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:37.682450 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:37.682479 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:37.682715 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:37.682877 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 22:24:37.683095 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":346,"threads_started":1}
I20260811 22:24:37.688793 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:37.692072 2771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6: Bootstrap starting.
I20260811 22:24:37.695639 2771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:37.695995 2775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:37.693349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48311 (local address 127.29.229.62:40579)
0811 22:24:37.693559 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:37.693567 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:37.693939 (+ 372us) server_negotiation.cc:407] Connection header received
0811 22:24:37.694155 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:37.694159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:37.694209 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:37.694321 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:37.695298 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:37.695304 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:37.695308 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:37.695326 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:37.695372 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:37.695387 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:37.695528 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:37.695646 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 22:24:37.695707 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:24:37.697472 2771 log.cc:824] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:37.700784 2771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6: No bootstrap required, opened a new log
I20260811 22:24:37.708415 2771 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01249faff3a443f8bf06af81e28982e6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40579 } }
I20260811 22:24:37.708890 2771 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:37.708969 2771 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01249faff3a443f8bf06af81e28982e6, State: Initialized, Role: FOLLOWER
I20260811 22:24:37.709493 2771 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01249faff3a443f8bf06af81e28982e6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40579 } }
I20260811 22:24:37.709692 2771 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:37.709795 2771 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:37.709928 2771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:37.712430 2771 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01249faff3a443f8bf06af81e28982e6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40579 } }
I20260811 22:24:37.713009 2771 leader_election.cc:302] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01249faff3a443f8bf06af81e28982e6; no voters:
I20260811 22:24:37.713558 2771 leader_election.cc:288] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:37.713719 2790 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:37.714452 2790 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6 [term 1 LEADER]: Becoming Leader. State: Replica: 01249faff3a443f8bf06af81e28982e6, State: Running, Role: LEADER
I20260811 22:24:37.714982 2790 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01249faff3a443f8bf06af81e28982e6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40579 } }
I20260811 22:24:37.715741 2771 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:37.718160 2792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01249faff3a443f8bf06af81e28982e6. Latest consensus state: current_term: 1 leader_uuid: "01249faff3a443f8bf06af81e28982e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01249faff3a443f8bf06af81e28982e6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40579 } } }
I20260811 22:24:37.720120 2792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:37.719385 2791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01249faff3a443f8bf06af81e28982e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01249faff3a443f8bf06af81e28982e6" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40579 } } }
I20260811 22:24:37.721788 2791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:37.723351 2803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:37.728778 2803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:37.739387 2803 catalog_manager.cc:1347] Generated new cluster ID: 4dc78a5b6b7e4050ae6b32520eabe0fd
I20260811 22:24:37.739544 2803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:37.754230 2803 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:37.754729 2803 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:37.765762 2803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01249faff3a443f8bf06af81e28982e6: Generated new TSK 0
I20260811 22:24:37.766943 2803 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:37.780731 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 2647
W20260811 22:24:37.781955 2765 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48329
W20260811 22:24:37.782694 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 2647: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:24:37.782862 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 2647
2026-08-11T22:24:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2484 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:37 dist-test-slave-jq4t krb5kdc[2824](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:37 dist-test-slave-jq4t krb5kdc[2824](info): set up 2 sockets
Aug 11 22:24:37 dist-test-slave-jq4t krb5kdc[2824](info): commencing operation
krb5kdc: starting...
W20260811 22:24:39.207919 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.376s user 0.001s sys 0.072s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:39 dist-test-slave-jq4t krb5kdc[2824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:24:39.645979 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:33213
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:39855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:33213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:40.032256 2847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:40.032747 2847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:40.032933 2847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:40.041389 2847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:40.041608 2847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:40.041707 2847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:40.041810 2847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:40.041894 2847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:40.053059 2847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:39855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:33213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:33213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:40.054600 2847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:40.056078 2847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:40.070657 2861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:40.070613 2866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:40.071167 2862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:40.072038 2847 server_base.cc:1034] running on GCE node
I20260811 22:24:40.073098 2847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:40.074787 2847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:40.076069 2847 hybrid_clock.cc:648] HybridClock initialized: now 1786487080076005 us; error 63 us; skew 500 ppm
I20260811 22:24:40.076545 2847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:40 dist-test-slave-jq4t krb5kdc[2824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487080, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:40.080233 2847 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:24:40.082464 2847 webserver.cc:460] Webserver started at http://127.29.229.62:34877/ using document root <none> and password file <none>
I20260811 22:24:40.083817 2847 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:40.084040 2847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:40.084410 2847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:40.087781 2847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "34f6d6db4413454ba64df81a3990222e"
format_stamp: "Formatted at 2026-08-11 22:24:40 on dist-test-slave-jq4t"
server_key: "c47638819c0493a83885f7fdfff223e0"
server_key_iv: "3e5bee785fbc32cfd8b1c8ad264d49d5"
server_key_version: "encryptionkey@0"
I20260811 22:24:40.088718 2847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "34f6d6db4413454ba64df81a3990222e"
format_stamp: "Formatted at 2026-08-11 22:24:40 on dist-test-slave-jq4t"
server_key: "c47638819c0493a83885f7fdfff223e0"
server_key_iv: "3e5bee785fbc32cfd8b1c8ad264d49d5"
server_key_version: "encryptionkey@0"
I20260811 22:24:40.094715 2847 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 22:24:40.099251 2878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:40.101263 2847 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 22:24:40.101495 2847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "34f6d6db4413454ba64df81a3990222e"
format_stamp: "Formatted at 2026-08-11 22:24:40 on dist-test-slave-jq4t"
server_key: "c47638819c0493a83885f7fdfff223e0"
server_key_iv: "3e5bee785fbc32cfd8b1c8ad264d49d5"
server_key_version: "encryptionkey@0"
I20260811 22:24:40.101739 2847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:40.127416 2847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:40.132051 2847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:40.132547 2847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:40.155376 2847 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:33213
I20260811 22:24:40.155400 2961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:33213 every 8 connection(s)
I20260811 22:24:40.156955 2847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:40.160144 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 2847
I20260811 22:24:40.160342 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:40.160997 30612 external_mini_cluster.cc:1428] Setting key ee5c12abb62eb98212afddd7d5d809ca
I20260811 22:24:40.161329 2966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:40.178242 2966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e: Bootstrap starting.
Aug 11 22:24:40 dist-test-slave-jq4t krb5kdc[2824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:24:40.182170 2966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:40.183681 2966 log.cc:824] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:40.188611 2966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e: No bootstrap required, opened a new log
I20260811 22:24:40.190657 2970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:40.162966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37663 (local address 127.29.229.62:33213)
0811 22:24:40.163816 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:40.163829 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:40.163870 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:24:40.167129 (+ 3259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:40.167156 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:40.167600 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:40.167954 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:40.170008 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:40.171375 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:40.173250 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:40.174031 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:40.174179 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:40.180338 (+ 6159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:40.180349 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:40.180368 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:40.180418 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:40.184712 (+ 4294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:40.186031 (+ 1319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:40.186225 (+ 194us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:40.186241 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:40.186587 (+ 346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:40.187159 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:40.187165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:40.187169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:40.187954 (+ 785us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:40.188275 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:40.188795 (+ 520us) server_negotiation.cc:299] Negotiation successful
0811 22:24:40.189298 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":391,"threads_started":1}
I20260811 22:24:40.195034 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:40.199350 2966 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f6d6db4413454ba64df81a3990222e" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 33213 } }
I20260811 22:24:40.199882 2966 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:40.199997 2966 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34f6d6db4413454ba64df81a3990222e, State: Initialized, Role: FOLLOWER
I20260811 22:24:40.200753 2966 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f6d6db4413454ba64df81a3990222e" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 33213 } }
I20260811 22:24:40.201050 2966 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:40.201215 2966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:40.201402 2966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:40.203766 2966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f6d6db4413454ba64df81a3990222e" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 33213 } }
I20260811 22:24:40.204421 2966 leader_election.cc:302] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34f6d6db4413454ba64df81a3990222e; no voters:
I20260811 22:24:40.205180 2966 leader_election.cc:288] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:40.205338 2990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:40.206050 2990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e [term 1 LEADER]: Becoming Leader. State: Replica: 34f6d6db4413454ba64df81a3990222e, State: Running, Role: LEADER
I20260811 22:24:40.206738 2990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f6d6db4413454ba64df81a3990222e" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 33213 } }
I20260811 22:24:40.209316 2966 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:40.209882 2994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34f6d6db4413454ba64df81a3990222e. Latest consensus state: current_term: 1 leader_uuid: "34f6d6db4413454ba64df81a3990222e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f6d6db4413454ba64df81a3990222e" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 33213 } } }
I20260811 22:24:40.211479 2994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:40.210476 2995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34f6d6db4413454ba64df81a3990222e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f6d6db4413454ba64df81a3990222e" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 33213 } } }
I20260811 22:24:40.212085 2995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:40.213004 3003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:40.220021 3003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:40.220436 2970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:40.200597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41455 (local address 127.29.229.62:33213)
0811 22:24:40.200800 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:40.200806 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:40.201401 (+ 595us) server_negotiation.cc:407] Connection header received
0811 22:24:40.201616 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:40.201622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:40.201715 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:40.201825 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:40.203904 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:40.205073 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:40.207216 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:40.207980 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:40.208114 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:40.211910 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:40.211921 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:40.211940 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:40.211994 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:40.216151 (+ 4157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:40.217175 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:40.217181 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:40.217189 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:40.217349 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:40.217847 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:40.217854 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:40.217858 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:40.218307 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:40.218591 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:40.219803 (+ 1212us) server_negotiation.cc:299] Negotiation successful
0811 22:24:40.220047 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:24:40.232879 3003 catalog_manager.cc:1347] Generated new cluster ID: 2059bb348f8a48ff8ac434e14fa12b53
I20260811 22:24:40.233037 3003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:40.253916 3003 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:40.255977 3003 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:40.268388 3003 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34f6d6db4413454ba64df81a3990222e: Generated new TSK 0
I20260811 22:24:40.269573 3003 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:40.277938 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 2847
I20260811 22:24:40.279603 2957 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:24:40.382364 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 2847
2026-08-11T22:24:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2602 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:40 dist-test-slave-jq4t krb5kdc[3037](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:40 dist-test-slave-jq4t krb5kdc[3037](info): set up 2 sockets
Aug 11 22:24:40 dist-test-slave-jq4t krb5kdc[3037](info): commencing operation
krb5kdc: starting...
W20260811 22:24:41.838488 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.404s user 0.003s sys 0.067s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:42 dist-test-slave-jq4t krb5kdc[3037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:24:42.271987 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:35413
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:35009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:35413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:42.641449 3057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:42.641762 3057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:42.641858 3057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:42.649191 3057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:42.649338 3057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:42.649389 3057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:42.649446 3057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:42.649492 3057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:42.659660 3057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:35009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:35413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:35413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:42.661039 3057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:42.662343 3057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:42.675365 3069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:42.675438 3073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:42.675320 3070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:42.681421 3057 server_base.cc:1034] running on GCE node
I20260811 22:24:42.682922 3057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:42.685715 3057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:42.686985 3057 hybrid_clock.cc:648] HybridClock initialized: now 1786487082686936 us; error 54 us; skew 500 ppm
I20260811 22:24:42.687388 3057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:42 dist-test-slave-jq4t krb5kdc[3037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487082, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:42.695916 3057 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:24:42.697847 3057 webserver.cc:460] Webserver started at http://127.29.229.62:42107/ using document root <none> and password file <none>
I20260811 22:24:42.698679 3057 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:42.698846 3057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:42.699183 3057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:42.702472 3057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "c9ad1cc47a6d4aadb93719f68ad7817f"
format_stamp: "Formatted at 2026-08-11 22:24:42 on dist-test-slave-jq4t"
server_key: "77a20fe8c7b69bfafcd1839d7c2daf48"
server_key_iv: "882d2e197ed4d7e46f461cf05124a3d7"
server_key_version: "encryptionkey@0"
I20260811 22:24:42.703290 3057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "c9ad1cc47a6d4aadb93719f68ad7817f"
format_stamp: "Formatted at 2026-08-11 22:24:42 on dist-test-slave-jq4t"
server_key: "77a20fe8c7b69bfafcd1839d7c2daf48"
server_key_iv: "882d2e197ed4d7e46f461cf05124a3d7"
server_key_version: "encryptionkey@0"
I20260811 22:24:42.709224 3057 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 22:24:42.712958 3090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:42.714450 3057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:24:42.714648 3057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "c9ad1cc47a6d4aadb93719f68ad7817f"
format_stamp: "Formatted at 2026-08-11 22:24:42 on dist-test-slave-jq4t"
server_key: "77a20fe8c7b69bfafcd1839d7c2daf48"
server_key_iv: "882d2e197ed4d7e46f461cf05124a3d7"
server_key_version: "encryptionkey@0"
I20260811 22:24:42.714864 3057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:42.757962 3057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:42.763187 3057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:42.763535 3057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:42.785480 3057 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:35413
I20260811 22:24:42.785499 3186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:35413 every 8 connection(s)
I20260811 22:24:42.786913 3057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:42.790814 3187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:42.796202 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 3057
I20260811 22:24:42.796406 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:42.796901 30612 external_mini_cluster.cc:1428] Setting key 5d8825c2ed9cb1d0d6fba9b756078562
I20260811 22:24:42.802145 3187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f: Bootstrap starting.
I20260811 22:24:42.806041 3187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:42.807301 3187 log.cc:824] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:42.810828 3187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f: No bootstrap required, opened a new log
I20260811 22:24:42.811614 3192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:42.798761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45673 (local address 127.29.229.62:35413)
0811 22:24:42.799690 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:42.799703 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:42.799751 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:24:42.801541 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:42.801566 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:42.801971 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:42.802340 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:42.803048 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:42.804465 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:42.806470 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:42.807199 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:42.807350 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:42.808902 (+ 1552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:42.808915 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:42.808937 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:42.809503 (+ 566us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:42.809576 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:42.809602 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:42.809703 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:42.810277 (+ 574us) server_negotiation.cc:299] Negotiation successful
0811 22:24:42.810535 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":705,"thread_start_us":473,"threads_started":1}
I20260811 22:24:42.815560 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:42.820969 3187 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ad1cc47a6d4aadb93719f68ad7817f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 35413 } }
I20260811 22:24:42.821550 3187 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:42.821676 3187 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ad1cc47a6d4aadb93719f68ad7817f, State: Initialized, Role: FOLLOWER
I20260811 22:24:42.822413 3187 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ad1cc47a6d4aadb93719f68ad7817f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 35413 } }
I20260811 22:24:42.822723 3187 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:42.822886 3187 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:42.823110 3187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:42.826073 3187 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ad1cc47a6d4aadb93719f68ad7817f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 35413 } }
I20260811 22:24:42.826717 3187 leader_election.cc:302] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9ad1cc47a6d4aadb93719f68ad7817f; no voters:
I20260811 22:24:42.827435 3187 leader_election.cc:288] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:42.827551 3205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:42.828851 3192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:42.821166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48847 (local address 127.29.229.62:35413)
0811 22:24:42.821370 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:42.821379 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:42.821970 (+ 591us) server_negotiation.cc:407] Connection header received
0811 22:24:42.822191 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:42.822196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:42.822280 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:42.822400 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:42.822987 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:42.824110 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:42.825465 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:42.826207 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:42.826337 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:42.827649 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:42.827655 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:42.827661 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:42.827685 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:42.827745 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:42.827774 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:42.827882 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:42.828395 (+ 513us) server_negotiation.cc:299] Negotiation successful
0811 22:24:42.828515 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:24:42.829905 3187 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:42.831847 3205 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f [term 1 LEADER]: Becoming Leader. State: Replica: c9ad1cc47a6d4aadb93719f68ad7817f, State: Running, Role: LEADER
I20260811 22:24:42.832728 3205 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ad1cc47a6d4aadb93719f68ad7817f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 35413 } }
I20260811 22:24:42.836046 3209 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9ad1cc47a6d4aadb93719f68ad7817f. Latest consensus state: current_term: 1 leader_uuid: "c9ad1cc47a6d4aadb93719f68ad7817f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ad1cc47a6d4aadb93719f68ad7817f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 35413 } } }
I20260811 22:24:42.836460 3209 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:42.836824 3207 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9ad1cc47a6d4aadb93719f68ad7817f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ad1cc47a6d4aadb93719f68ad7817f" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 35413 } } }
I20260811 22:24:42.837246 3207 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f [sys.catalog]: This master's current role is: LEADER
W20260811 22:24:42.846129 3244 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:24:42.846311 3244 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:24:42.847615 3240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:42.849691 3240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:42.859050 3240 catalog_manager.cc:1347] Generated new cluster ID: e13bc35cb54d48ccbe19c7863d753e8f
I20260811 22:24:42.859193 3240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:42.872056 3240 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:42.874014 3240 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:42.883829 3240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9ad1cc47a6d4aadb93719f68ad7817f: Generated new TSK 0
I20260811 22:24:42.884799 3240 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:42.896299 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 3057
W20260811 22:24:42.897675 3178 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45673
W20260811 22:24:42.898435 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 3057: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:24:42.898588 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 3057
2026-08-11T22:24:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:43 dist-test-slave-jq4t krb5kdc[3254](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:43 dist-test-slave-jq4t krb5kdc[3254](info): set up 2 sockets
Aug 11 22:24:43 dist-test-slave-jq4t krb5kdc[3254](info): commencing operation
krb5kdc: starting...
W20260811 22:24:44.363065 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.414s user 0.007s sys 0.065s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:44 dist-test-slave-jq4t krb5kdc[3254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:24:44.808178 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:34697
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:35193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:34697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:45.204407 3277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:45.205027 3277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:45.205228 3277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:45.214582 3277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:45.214805 3277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:45.214946 3277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:45.215068 3277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:45.215150 3277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:45.225751 3277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:35193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:34697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:34697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:45.227274 3277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:45.228756 3277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:45.242252 3284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:45.242331 3289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:45.242308 3286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:45.243355 3277 server_base.cc:1034] running on GCE node
I20260811 22:24:45.244764 3277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:45.246378 3277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:45.247623 3277 hybrid_clock.cc:648] HybridClock initialized: now 1786487085247561 us; error 56 us; skew 500 ppm
I20260811 22:24:45.248075 3277 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:45 dist-test-slave-jq4t krb5kdc[3254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487085, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:45.251834 3277 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:24:45.254233 3277 webserver.cc:460] Webserver started at http://127.29.229.62:35189/ using document root <none> and password file <none>
I20260811 22:24:45.255040 3277 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:45.255155 3277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:45.255525 3277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:45.258913 3277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "07fcf85989fd47b6bdb64e1e8281f6e8"
format_stamp: "Formatted at 2026-08-11 22:24:45 on dist-test-slave-jq4t"
server_key: "b8ad6be99b588711501b7edfb7fd60e3"
server_key_iv: "39bc1017f8e026da6bb8814fcc059a3e"
server_key_version: "encryptionkey@0"
I20260811 22:24:45.259832 3277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "07fcf85989fd47b6bdb64e1e8281f6e8"
format_stamp: "Formatted at 2026-08-11 22:24:45 on dist-test-slave-jq4t"
server_key: "b8ad6be99b588711501b7edfb7fd60e3"
server_key_iv: "39bc1017f8e026da6bb8814fcc059a3e"
server_key_version: "encryptionkey@0"
I20260811 22:24:45.268273 3277 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260811 22:24:45.272967 3298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:45.274885 3277 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:24:45.275135 3277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "07fcf85989fd47b6bdb64e1e8281f6e8"
format_stamp: "Formatted at 2026-08-11 22:24:45 on dist-test-slave-jq4t"
server_key: "b8ad6be99b588711501b7edfb7fd60e3"
server_key_iv: "39bc1017f8e026da6bb8814fcc059a3e"
server_key_version: "encryptionkey@0"
I20260811 22:24:45.275360 3277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:45.282107 3277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:45.287176 3277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:45.287539 3277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:45.311252 3277 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:34697
I20260811 22:24:45.311299 3394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:34697 every 8 connection(s)
I20260811 22:24:45.312829 3277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:45.316954 3395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:45.321662 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 3277
I20260811 22:24:45.321938 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:45.322518 30612 external_mini_cluster.cc:1428] Setting key 928741c3b172ad3b7a3154f59dd74ac9
I20260811 22:24:45.330133 3395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8: Bootstrap starting.
I20260811 22:24:45.332643 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:45.324763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60387 (local address 127.29.229.62:34697)
0811 22:24:45.325648 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:45.325665 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:45.325722 (+ 57us) server_negotiation.cc:407] Connection header received
0811 22:24:45.328301 (+ 2579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:45.328334 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:45.328815 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:45.329281 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:45.330493 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:45.330506 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:45.330525 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:45.330547 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:45.330632 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:45.330663 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:45.330934 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:45.331067 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 22:24:45.331259 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":352,"threads_started":1}
I20260811 22:24:45.334707 3395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:45.336462 3395 log.cc:824] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:45.338136 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:45.339902 3395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8: No bootstrap required, opened a new log
I20260811 22:24:45.345928 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:45.342897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55513 (local address 127.29.229.62:34697)
0811 22:24:45.343094 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:45.343100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:45.343488 (+ 388us) server_negotiation.cc:407] Connection header received
0811 22:24:45.343646 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:45.343651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:45.343718 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:45.343842 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:45.344965 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:45.344975 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:45.344980 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:24:45.344999 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:45.345047 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:24:45.345066 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:45.345200 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:45.345541 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 22:24:45.345605 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:24:45.349349 3395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fcf85989fd47b6bdb64e1e8281f6e8" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 34697 } }
I20260811 22:24:45.349736 3395 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:24:45.349802 3395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07fcf85989fd47b6bdb64e1e8281f6e8, State: Initialized, Role: FOLLOWER
I20260811 22:24:45.350468 3395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fcf85989fd47b6bdb64e1e8281f6e8" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 34697 } }
I20260811 22:24:45.350726 3395 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:24:45.350914 3395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:24:45.351118 3395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:45.354374 3395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fcf85989fd47b6bdb64e1e8281f6e8" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 34697 } }
I20260811 22:24:45.355233 3395 leader_election.cc:302] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07fcf85989fd47b6bdb64e1e8281f6e8; no voters:
I20260811 22:24:45.356077 3395 leader_election.cc:288] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:24:45.356355 3409 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:45.358119 3409 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8 [term 1 LEADER]: Becoming Leader. State: Replica: 07fcf85989fd47b6bdb64e1e8281f6e8, State: Running, Role: LEADER
I20260811 22:24:45.358907 3409 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fcf85989fd47b6bdb64e1e8281f6e8" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 34697 } }
I20260811 22:24:45.359141 3395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:45.361469 3412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07fcf85989fd47b6bdb64e1e8281f6e8. Latest consensus state: current_term: 1 leader_uuid: "07fcf85989fd47b6bdb64e1e8281f6e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fcf85989fd47b6bdb64e1e8281f6e8" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 34697 } } }
I20260811 22:24:45.361855 3412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:45.362502 3410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07fcf85989fd47b6bdb64e1e8281f6e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fcf85989fd47b6bdb64e1e8281f6e8" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 34697 } } }
I20260811 22:24:45.362802 3410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:45.362973 3425 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:45.369964 3425 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:45.390260 3425 catalog_manager.cc:1347] Generated new cluster ID: 1e554323a30a44f8ad24b66aa1ad0ea2
I20260811 22:24:45.390872 3425 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:45.405192 3425 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:45.405761 3425 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:45.425351 3425 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07fcf85989fd47b6bdb64e1e8281f6e8: Generated new TSK 0
I20260811 22:24:45.426452 3425 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:45.436432 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 3277
W20260811 22:24:45.438472 3382 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60387
W20260811 22:24:45.439194 30612 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 3277: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:24:45.439325 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 3277
2026-08-11T22:24:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2536 ms)
[----------] 18 tests from AuthTokenIssuingTest (45307 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:45 dist-test-slave-jq4t krb5kdc[3449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:45 dist-test-slave-jq4t krb5kdc[3449](info): set up 2 sockets
Aug 11 22:24:45 dist-test-slave-jq4t krb5kdc[3449](info): commencing operation
krb5kdc: starting...
W20260811 22:24:46.848881 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.363s user 0.000s sys 0.064s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:47 dist-test-slave-jq4t krb5kdc[3449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:24:47.277334 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:44339
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:35661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:44339,127.29.229.61:45157,127.29.229.60:39613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:47.685457 3476 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:24:47.685806 3476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:47.685889 3476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:47.685981 3476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:47.693672 3476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:47.693776 3476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:47.693836 3476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:47.693897 3476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:47.693956 3476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:47.704667 3476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:35661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:44339,127.29.229.61:45157,127.29.229.60:39613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:44339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:47.706010 3476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:47.707309 3476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:47.720456 3490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:47.720754 3493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:47.720571 3488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:47.721956 3476 server_base.cc:1034] running on GCE node
I20260811 22:24:47.722837 3476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:47.724362 3476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:47.725647 3476 hybrid_clock.cc:648] HybridClock initialized: now 1786487087725580 us; error 39 us; skew 500 ppm
I20260811 22:24:47.726032 3476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:47 dist-test-slave-jq4t krb5kdc[3449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487087, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:47.732219 3476 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:24:47.734229 3476 webserver.cc:460] Webserver started at http://127.29.229.62:44539/ using document root <none> and password file <none>
I20260811 22:24:47.735026 3476 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:47.735181 3476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:47.735561 3476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:47.738997 3476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "273c0566d3ab47e6a2a602a07fd83191"
format_stamp: "Formatted at 2026-08-11 22:24:47 on dist-test-slave-jq4t"
server_key: "5cbf3bdcda1c55f1733bdfd1cd65f566"
server_key_iv: "60052951974cb8543cccb4841019c523"
server_key_version: "encryptionkey@0"
I20260811 22:24:47.740430 3476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "273c0566d3ab47e6a2a602a07fd83191"
format_stamp: "Formatted at 2026-08-11 22:24:47 on dist-test-slave-jq4t"
server_key: "5cbf3bdcda1c55f1733bdfd1cd65f566"
server_key_iv: "60052951974cb8543cccb4841019c523"
server_key_version: "encryptionkey@0"
I20260811 22:24:47.748229 3476 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260811 22:24:47.752663 3506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:47.754559 3476 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:24:47.754873 3476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "273c0566d3ab47e6a2a602a07fd83191"
format_stamp: "Formatted at 2026-08-11 22:24:47 on dist-test-slave-jq4t"
server_key: "5cbf3bdcda1c55f1733bdfd1cd65f566"
server_key_iv: "60052951974cb8543cccb4841019c523"
server_key_version: "encryptionkey@0"
I20260811 22:24:47.755203 3476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:47.773841 3476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:47.778484 3476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:47.778797 3476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:47.803647 3476 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:44339
I20260811 22:24:47.803660 3614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:44339 every 8 connection(s)
I20260811 22:24:47.805382 3476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:47.809574 3615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:47.812638 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 3476
I20260811 22:24:47.812882 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:47.813745 30612 external_mini_cluster.cc:1428] Setting key 769511f6f0367fdb5911f5fbe74fdf4c
I20260811 22:24:47.819562 3615 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:24:47 dist-test-slave-jq4t krb5kdc[3449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487087, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:24:47.845667 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:47.823908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56240 (local address 127.29.229.62:44339)
0811 22:24:47.825773 (+ 1865us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:47.825781 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:47.825844 (+ 63us) server_negotiation.cc:407] Connection header received
0811 22:24:47.827623 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:47.827629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:47.827881 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:47.828102 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:47.829539 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:47.830762 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:47.832205 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:47.832851 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:47.833068 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:47.837634 (+ 4566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:47.837642 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:47.837656 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:47.837691 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:47.840766 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:47.841564 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:47.841577 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:47.841579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:47.841738 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:47.842159 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:47.842166 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:47.842169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:47.842711 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:47.843049 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:47.843679 (+ 630us) server_negotiation.cc:299] Negotiation successful
0811 22:24:47.844064 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":1628,"thread_start_us":1442,"threads_started":1}
I20260811 22:24:47.845703 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:47.823908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:44339 (local address 127.0.0.1:56240)
0811 22:24:47.825016 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:47.825068 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:47.827196 (+ 2128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:47.828124 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:47.828130 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:47.828944 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:47.829297 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:47.829313 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:47.830958 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:47.830965 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:47.832056 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:47.832068 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:47.833121 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:47.833128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:47.833412 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:47.834143 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:47.834180 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:47.837357 (+ 3177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:47.840962 (+ 3605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:47.840974 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:47.840987 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:47.841411 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:47.841859 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:47.841865 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:47.841868 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:47.842039 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:47.842866 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:47.842985 (+ 119us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:47.843253 (+ 268us) client_negotiation.cc:769] Sending connection context
0811 22:24:47.843546 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 22:24:47.844194 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":252,"thread_start_us":436,"threads_started":1}
I20260811 22:24:47.849785 3615 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:24:47.851353 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:47.850849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45157 (local address 127.0.0.1:51138)
0811 22:24:47.851066 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:47.851215 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260811 22:24:47.851666 3510 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45157: connect: Connection refused (error 111)
I20260811 22:24:47.853518 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:47.853118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45157 (local address 127.0.0.1:51152)
0811 22:24:47.853287 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:47.853357 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 22:24:47.853720 3615 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.61:45157: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45157: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.29.229.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.61@KRBTEST.COM" created.
I20260811 22:24:47.903137 3615 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } attempt: 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.61.keytab.
Entry for principal kudu/127.29.229.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.61.keytab.
I20260811 22:24:47.904541 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:47.904030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45157 (local address 127.0.0.1:51158)
0811 22:24:47.904238 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:47.904359 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 22:24:47.904903 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.61:45157
--webserver_interface=127.29.229.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.61.keytab
--principal=kudu/127.29.229.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:35661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:44339,127.29.229.61:45157,127.29.229.60:39613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:24:47.905706 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:47.905339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45157 (local address 127.0.0.1:51168)
0811 22:24:47.905495 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:47.905575 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 22:24:47.905937 3615 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.61:45157: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45157: connect: Connection refused (error 111)
I20260811 22:24:47.974270 3615 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } attempt: 2
I20260811 22:24:47.975623 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:47.975037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45157 (local address 127.0.0.1:51170)
0811 22:24:47.975325 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:47.975445 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 22:24:47.976823 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:47.976366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45157 (local address 127.0.0.1:51186)
0811 22:24:47.976584 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:47.976667 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 22:24:47.977063 3615 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.61:45157: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45157: connect: Connection refused (error 111)
I20260811 22:24:48.068403 3615 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } attempt: 3
I20260811 22:24:48.069816 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.069283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45157 (local address 127.0.0.1:51202)
0811 22:24:48.069538 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.069653 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 22:24:48.071027 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.070559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45157 (local address 127.0.0.1:51206)
0811 22:24:48.070721 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.070828 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 22:24:48.071256 3615 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.61:45157: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45157: connect: Connection refused (error 111)
I20260811 22:24:48.214542 3615 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } attempt: 4
I20260811 22:24:48.216269 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.215318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45157 (local address 127.0.0.1:51214)
0811 22:24:48.215678 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.215801 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183}
I20260811 22:24:48.217718 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.217316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45157 (local address 127.0.0.1:51230)
0811 22:24:48.217475 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.217567 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 22:24:48.217989 3615 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.61:45157: Network error: Client connection negotiation failed: client connection to 127.29.229.61:45157: connect: Connection refused (error 111)
W20260811 22:24:48.281270 3628 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:24:48.281674 3628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:48.281826 3628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:48.281921 3628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:48.289568 3628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:48.289672 3628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:48.289769 3628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:48.289870 3628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:48.290000 3628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:48.300940 3628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:35661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:44339,127.29.229.61:45157,127.29.229.60:39613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.61.keytab
--principal=kudu/127.29.229.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.61:45157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.229.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:48.302423 3628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:48.303891 3628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:48.316869 3633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:48.316937 3634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:48.317111 3640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:48.318009 3628 server_base.cc:1034] running on GCE node
I20260811 22:24:48.318902 3628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:48.320523 3628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:48.321803 3628 hybrid_clock.cc:648] HybridClock initialized: now 1786487088321738 us; error 41 us; skew 500 ppm
I20260811 22:24:48.322172 3628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:48 dist-test-slave-jq4t krb5kdc[3449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487088, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:48.325573 3628 init.cc:377] Logged in from keytab as kudu/127.29.229.61@KRBTEST.COM (short username kudu)
I20260811 22:24:48.327188 3628 webserver.cc:460] Webserver started at http://127.29.229.61:37537/ using document root <none> and password file <none>
I20260811 22:24:48.327917 3628 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:48.328060 3628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:48.328406 3628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:48.331658 3628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-1/data/instance:
uuid: "4e0e3e24609a4b859260bf4ff0141521"
format_stamp: "Formatted at 2026-08-11 22:24:48 on dist-test-slave-jq4t"
server_key: "9a4d02072746f6b6c84a11d1e4cacea8"
server_key_iv: "c66c9726a6b19a0cc2b55b0031288aeb"
server_key_version: "encryptionkey@0"
I20260811 22:24:48.332571 3628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-1/wal/instance:
uuid: "4e0e3e24609a4b859260bf4ff0141521"
format_stamp: "Formatted at 2026-08-11 22:24:48 on dist-test-slave-jq4t"
server_key: "9a4d02072746f6b6c84a11d1e4cacea8"
server_key_iv: "c66c9726a6b19a0cc2b55b0031288aeb"
server_key_version: "encryptionkey@0"
I20260811 22:24:48.337808 3628 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 22:24:48.341527 3651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:48.342994 3628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 22:24:48.343200 3628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-1/wal
uuid: "4e0e3e24609a4b859260bf4ff0141521"
format_stamp: "Formatted at 2026-08-11 22:24:48 on dist-test-slave-jq4t"
server_key: "9a4d02072746f6b6c84a11d1e4cacea8"
server_key_iv: "c66c9726a6b19a0cc2b55b0031288aeb"
server_key_version: "encryptionkey@0"
I20260811 22:24:48.343451 3628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:48.387564 3628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:48.393185 3628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:48.393788 3628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:48.417308 3761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.61:45157 every 8 connection(s)
I20260811 22:24:48.417411 3628 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.61:45157
I20260811 22:24:48.419094 3628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-1/data/info.pb
I20260811 22:24:48.423156 3762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:48.426630 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 3628
I20260811 22:24:48.426828 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-1/wal/instance
I20260811 22:24:48.427394 30612 external_mini_cluster.cc:1428] Setting key b067282d0d6cdc9ce2603bfbcee0e482
I20260811 22:24:48.432528 3762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:24:48 dist-test-slave-jq4t krb5kdc[3449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487088, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.61@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:24:48.458938 3766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.436086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56246 (local address 127.29.229.62:44339)
0811 22:24:48.436693 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:48.436700 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:48.437779 (+ 1079us) server_negotiation.cc:407] Connection header received
0811 22:24:48.440820 (+ 3041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:48.440826 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:48.440911 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:48.441015 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:48.443309 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:48.444134 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:48.446342 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:48.447096 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:48.447243 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:48.450957 (+ 3714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:48.450969 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:48.450977 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:48.451052 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:48.455113 (+ 4061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:48.455975 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:48.455983 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:48.455989 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:48.456149 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:48.456711 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:48.456719 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:48.456725 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:48.457160 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:48.457318 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:48.458158 (+ 840us) server_negotiation.cc:299] Negotiation successful
0811 22:24:48.458427 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":300,"threads_started":1}
I20260811 22:24:48.459836 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.436681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:44339 (local address 127.0.0.1:56246)
0811 22:24:48.437571 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.437630 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:48.440461 (+ 2831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:48.441336 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:48.441350 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:48.442194 (+ 844us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:48.443057 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:48.443084 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:48.444315 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:48.444323 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:48.446183 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:48.446197 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:48.447323 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:48.447329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:48.447549 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:48.448327 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:48.448352 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:48.450660 (+ 2308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:48.455283 (+ 4623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:48.455290 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:48.455304 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:48.455812 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:48.456274 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:48.456278 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:48.456280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:48.456545 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:48.457319 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:48.457326 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:48.457703 (+ 377us) client_negotiation.cc:769] Sending connection context
0811 22:24:48.457982 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 22:24:48.458500 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":343,"threads_started":1}
I20260811 22:24:48.461947 3762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.229.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.60@KRBTEST.COM" created.
Aug 11 22:24:48 dist-test-slave-jq4t krb5kdc[3449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487088, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.61@KRBTEST.COM for kudu/127.29.229.61@KRBTEST.COM
I20260811 22:24:48.478374 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.462623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45157 (local address 127.0.0.1:51242)
0811 22:24:48.462799 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.462908 (+ 109us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:48.463028 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:48.464166 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:48.464171 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:48.464690 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:48.464836 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:48.464846 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:48.466291 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:48.466297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:48.467154 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:48.467166 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:48.468033 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:48.468039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:48.468275 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:48.468983 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:48.469002 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:48.472104 (+ 3102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:48.475662 (+ 3558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:48.475671 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:48.475675 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:48.476074 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:48.476515 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:48.476523 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:48.476527 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:48.476683 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:48.477295 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:48.477300 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:48.477446 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 22:24:48.477600 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 22:24:48.477926 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":6}
I20260811 22:24:48.478421 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.463019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51242 (local address 127.29.229.61:45157)
0811 22:24:48.463715 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:48.463721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:48.463748 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:24:48.463796 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:48.463801 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:48.463991 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:48.464181 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:48.465024 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:48.466144 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:48.467302 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:48.467875 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:48.468024 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:48.472354 (+ 4330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:48.472369 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:48.472383 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:48.472413 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:48.475498 (+ 3085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:48.476218 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:48.476229 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:48.476233 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:48.476361 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:48.476795 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:48.476802 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:48.476805 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:48.477164 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:48.477402 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:48.477771 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 22:24:48.478038 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":553,"thread_start_us":422,"threads_started":1}
I20260811 22:24:48.480901 3762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:24:48.482331 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.481679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51370)
0811 22:24:48.482026 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.482155 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 22:24:48.482723 3652 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
I20260811 22:24:48.484607 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.484100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51376)
0811 22:24:48.484294 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.484426 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260811 22:24:48.484886 3762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:39613: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
I20260811 22:24:48.517297 3615 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } attempt: 5
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.60.keytab.
Entry for principal kudu/127.29.229.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.60.keytab.
I20260811 22:24:48.522522 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.60:39613
--webserver_interface=127.29.229.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.60.keytab
--principal=kudu/127.29.229.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:35661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:44339,127.29.229.61:45157,127.29.229.60:39613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:24:48 dist-test-slave-jq4t krb5kdc[3449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487087, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for kudu/127.29.229.61@KRBTEST.COM
I20260811 22:24:48.533157 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.518157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45157 (local address 127.0.0.1:51254)
0811 22:24:48.518388 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.518413 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:48.518561 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:48.518977 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:48.518981 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:48.519378 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:48.519570 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:48.519579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:48.520874 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:48.520878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:48.521860 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:48.521873 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:48.522930 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:48.522935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:48.523127 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:48.523836 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:48.523857 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:48.526038 (+ 2181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:48.530343 (+ 4305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:48.530349 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:48.530353 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:48.530783 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:48.531288 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:48.531292 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:48.531295 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:48.531439 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:48.532218 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:48.532222 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:48.532371 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 22:24:48.532600 (+ 229us) client_negotiation.cc:240] Negotiation successful
0811 22:24:48.532882 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 22:24:48.533434 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.518422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51254 (local address 127.29.229.61:45157)
0811 22:24:48.518633 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:48.518638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:48.518702 (+ 64us) server_negotiation.cc:407] Connection header received
0811 22:24:48.518771 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:48.518775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:48.518856 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:48.518937 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:48.519724 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:48.520639 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:48.522034 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:48.522723 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:48.522878 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:48.526266 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:48.526275 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:48.526283 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:48.526331 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:48.530161 (+ 3830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:48.530961 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:48.530977 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:48.530983 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:48.531144 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:48.531612 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:48.531620 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:48.531626 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:48.532066 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:48.532235 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:48.532734 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 22:24:48.533029 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:24:48.534265 3762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } attempt: 1
I20260811 22:24:48.534528 3615 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:24:48.535367 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.534858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51382)
0811 22:24:48.535074 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.535195 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 22:24:48.535694 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.535248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51396)
0811 22:24:48.535416 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.535513 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 22:24:48.536592 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.536154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51406)
0811 22:24:48.536333 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.536421 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 22:24:48.536700 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.536302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51410)
0811 22:24:48.536446 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.536577 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 22:24:48.536870 3615 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:39613: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
W20260811 22:24:48.536998 3762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:39613: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
I20260811 22:24:48.588135 3615 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } attempt: 1
I20260811 22:24:48.589569 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.588974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51418)
0811 22:24:48.589216 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.589333 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 22:24:48.590736 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.590346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51422)
0811 22:24:48.590520 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.590609 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 22:24:48.591014 3615 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:39613: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
I20260811 22:24:48.605338 3762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } attempt: 2
I20260811 22:24:48.607040 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.606376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51430)
0811 22:24:48.606695 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.606829 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260811 22:24:48.608273 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.607877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51438)
0811 22:24:48.608057 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.608150 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 22:24:48.608572 3762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:39613: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
I20260811 22:24:48.659336 3615 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } attempt: 2
I20260811 22:24:48.660697 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.660127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51446)
0811 22:24:48.660374 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.660465 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 22:24:48.661737 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.661380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51448)
0811 22:24:48.661547 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.661623 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 22:24:48.661942 3615 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:39613: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
I20260811 22:24:48.699831 3762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } attempt: 3
I20260811 22:24:48.701390 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.700848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51456)
0811 22:24:48.701079 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.701190 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 22:24:48.702730 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.702207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51466)
0811 22:24:48.702498 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.702597 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
W20260811 22:24:48.702940 3762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:39613: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
I20260811 22:24:48.768229 3615 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } attempt: 3
I20260811 22:24:48.769927 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.769313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51468)
0811 22:24:48.769585 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.769705 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260811 22:24:48.771437 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.770866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51472)
0811 22:24:48.771169 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.771278 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
W20260811 22:24:48.771754 3615 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:39613: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
I20260811 22:24:48.846158 3762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } attempt: 4
I20260811 22:24:48.847435 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.846893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51478)
0811 22:24:48.847093 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.847179 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 22:24:48.848680 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.848295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51490)
0811 22:24:48.848449 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.848570 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 22:24:48.848903 3762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:39613: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
W20260811 22:24:48.921283 3774 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:24:48.921657 3774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:48.921784 3774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:48.921926 3774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:48.929503 3774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:48.929641 3774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:48.929765 3774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:48.929853 3774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:48.929971 3774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:48.940464 3774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:35661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:44339,127.29.229.61:45157,127.29.229.60:39613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.60.keytab
--principal=kudu/127.29.229.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.60:39613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.229.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:48.942404 3774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:48.943958 3774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:24:48.949117 3615 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } attempt: 4
I20260811 22:24:48.950544 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.949867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51506)
0811 22:24:48.950217 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.950326 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202}
I20260811 22:24:48.951766 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:48.951414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51510)
0811 22:24:48.951578 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:48.951647 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 22:24:48.952066 3615 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:39613: Network error: Client connection negotiation failed: client connection to 127.29.229.60:39613: connect: Connection refused (error 111)
W20260811 22:24:48.957422 3782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:48.957526 3780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:48.958177 3774 server_base.cc:1034] running on GCE node
W20260811 22:24:48.958235 3787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:48.959208 3774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:48.960899 3774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:48.962146 3774 hybrid_clock.cc:648] HybridClock initialized: now 1786487088962059 us; error 58 us; skew 500 ppm
I20260811 22:24:48.962483 3774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:48 dist-test-slave-jq4t krb5kdc[3449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487088, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:48.966527 3774 init.cc:377] Logged in from keytab as kudu/127.29.229.60@KRBTEST.COM (short username kudu)
I20260811 22:24:48.968232 3774 webserver.cc:460] Webserver started at http://127.29.229.60:37919/ using document root <none> and password file <none>
I20260811 22:24:48.969100 3774 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:48.969250 3774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:48.969612 3774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:48.973079 3774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-2/data/instance:
uuid: "e59f8a152339453d91a55e3ab3b9ff66"
format_stamp: "Formatted at 2026-08-11 22:24:48 on dist-test-slave-jq4t"
server_key: "cc39686942796d2f7fe0b99ea4dfaee8"
server_key_iv: "65b445c0b08e41f38a18f4e77f5d0f81"
server_key_version: "encryptionkey@0"
I20260811 22:24:48.973973 3774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-2/wal/instance:
uuid: "e59f8a152339453d91a55e3ab3b9ff66"
format_stamp: "Formatted at 2026-08-11 22:24:48 on dist-test-slave-jq4t"
server_key: "cc39686942796d2f7fe0b99ea4dfaee8"
server_key_iv: "65b445c0b08e41f38a18f4e77f5d0f81"
server_key_version: "encryptionkey@0"
I20260811 22:24:48.979643 3774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 22:24:48.983574 3798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:48.985234 3774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:24:48.985455 3774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-2/wal
uuid: "e59f8a152339453d91a55e3ab3b9ff66"
format_stamp: "Formatted at 2026-08-11 22:24:48 on dist-test-slave-jq4t"
server_key: "cc39686942796d2f7fe0b99ea4dfaee8"
server_key_iv: "65b445c0b08e41f38a18f4e77f5d0f81"
server_key_version: "encryptionkey@0"
I20260811 22:24:48.985705 3774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:49.019692 3774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:49.024384 3774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:49.024744 3774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:49.045176 3774 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.60:39613
I20260811 22:24:49.045202 3890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.60:39613 every 8 connection(s)
I20260811 22:24:49.046762 3774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-2/data/info.pb
I20260811 22:24:49.047334 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 3774
I20260811 22:24:49.047566 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486685119861-30612-0/minicluster-data/master-2/wal/instance
I20260811 22:24:49.048206 30612 external_mini_cluster.cc:1428] Setting key e61342436853470555ca93b48ef584c2
I20260811 22:24:49.052116 3891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:49.061988 3891 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:24:49 dist-test-slave-jq4t krb5kdc[3449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.60@KRBTEST.COM
Aug 11 22:24:49 dist-test-slave-jq4t krb5kdc[3449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487088, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.60@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:24:49.075608 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:49.050333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51516 (local address 127.29.229.60:39613)
0811 22:24:49.051197 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.051214 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.051265 (+ 51us) server_negotiation.cc:407] Connection header received
0811 22:24:49.053851 (+ 2586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.053884 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.054414 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.054897 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.056466 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.058804 (+ 2338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.060307 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.061446 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.061630 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:49.067179 (+ 5549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:49.067188 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:49.067199 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:49.067245 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:49.070837 (+ 3592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.071701 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.071711 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.071721 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.071919 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.072371 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.072377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.072380 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.073165 (+ 785us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:49.073553 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:49.074056 (+ 503us) server_negotiation.cc:299] Negotiation successful
0811 22:24:49.074528 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"spinlock_wait_cycles":10624,"thread_start_us":372,"threads_started":1}
I20260811 22:24:49.081413 3896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:49.064809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56262 (local address 127.29.229.62:44339)
0811 22:24:49.065365 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.065372 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.066254 (+ 882us) server_negotiation.cc:407] Connection header received
0811 22:24:49.066598 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.066603 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.066671 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.066775 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.068314 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.069493 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.071092 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.071505 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.071631 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:49.075152 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:49.075157 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:49.075160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:49.075203 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:49.078048 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.078952 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.078958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.078963 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.079108 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.079602 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.079607 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.079610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.080077 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:49.080275 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:49.080909 (+ 634us) server_negotiation.cc:299] Negotiation successful
0811 22:24:49.081107 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":273,"threads_started":1}
I20260811 22:24:49.081648 3899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:49.064880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:44339 (local address 127.0.0.1:56262)
0811 22:24:49.065783 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:49.066043 (+ 260us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:49.066422 (+ 379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:49.066959 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:49.066968 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:49.067704 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:49.068113 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:49.068130 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.069698 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.069702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:49.070931 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:49.070950 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.071712 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.071717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:49.071853 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:49.072628 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:49.072645 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:49.074933 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:49.078209 (+ 3276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:49.078223 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:49.078230 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:49.078819 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:49.079229 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:49.079234 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:49.079238 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:49.079448 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:49.080244 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:49.080252 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:49.080444 (+ 192us) client_negotiation.cc:769] Sending connection context
0811 22:24:49.080735 (+ 291us) client_negotiation.cc:240] Negotiation successful
0811 22:24:49.081119 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"spinlock_wait_cycles":14720,"thread_start_us":476,"threads_started":1}
I20260811 22:24:49.083580 3891 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:24:49 dist-test-slave-jq4t krb5kdc[3449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
Aug 11 22:24:49 dist-test-slave-jq4t krb5kdc[3449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487088, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.60@KRBTEST.COM for kudu/127.29.229.61@KRBTEST.COM
I20260811 22:24:49.100875 3906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:49.079683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56270 (local address 127.29.229.62:44339)
0811 22:24:49.080406 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.080410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.080429 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:24:49.080466 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.080470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.080598 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.080682 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.082504 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.083623 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.085929 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.086892 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.087075 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:49.092925 (+ 5850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:49.092933 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:49.092938 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:49.092987 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:49.096403 (+ 3416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.097743 (+ 1340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.097748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.097750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.097991 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.098615 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.098619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.098621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.098948 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:49.099192 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:49.099860 (+ 668us) server_negotiation.cc:299] Negotiation successful
0811 22:24:49.100301 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":399,"threads_started":1}
I20260811 22:24:49.103643 3899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:49.084942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:45157 (local address 127.0.0.1:51262)
0811 22:24:49.085296 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:49.085330 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:49.085488 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:49.086359 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:49.086366 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:49.087205 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:49.087433 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:49.087450 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.089314 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.089318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:49.090537 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:49.090557 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.091726 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.091739 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:49.091894 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:49.092605 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:49.092623 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:49.095371 (+ 2748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:49.100033 (+ 4662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:49.100039 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:49.100043 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:49.100738 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:49.101401 (+ 663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:49.101406 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:49.101409 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:49.101604 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:49.102486 (+ 882us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:49.102492 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:49.102731 (+ 239us) client_negotiation.cc:769] Sending connection context
0811 22:24:49.102894 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 22:24:49.103274 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 22:24:49.103958 3908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:49.085204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51262 (local address 127.29.229.61:45157)
0811 22:24:49.085988 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.085993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.086012 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:24:49.086072 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.086076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.086146 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.086256 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.087891 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.089159 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.090711 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.091505 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.091658 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:49.095604 (+ 3946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:49.095610 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:49.095613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:49.095649 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:49.099802 (+ 4153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.100935 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.100942 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.100944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.101222 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.101858 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.101862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.101864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.102280 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:49.102492 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:49.103087 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 22:24:49.103403 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":508,"threads_started":1}
I20260811 22:24:49.105614 3891 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:24:49 dist-test-slave-jq4t krb5kdc[3449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487088, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.60@KRBTEST.COM for kudu/127.29.229.60@KRBTEST.COM
Aug 11 22:24:49 dist-test-slave-jq4t krb5kdc[3449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.61@KRBTEST.COM
I20260811 22:24:49.120797 3899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:49.106960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51528)
0811 22:24:49.107109 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:49.107130 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:49.107278 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:49.107581 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:49.107585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:49.108019 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:49.108170 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:49.108179 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.109222 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.109228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:49.110083 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:49.110095 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.111230 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.111239 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:49.111419 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:49.111998 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:49.112015 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:49.114395 (+ 2380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:49.118143 (+ 3748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:49.118147 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:49.118149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:49.118541 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:49.119021 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:49.119027 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:49.119030 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:49.119229 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:49.119780 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:49.119784 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:49.119922 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 22:24:49.120119 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 22:24:49.120442 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 22:24:49.121316 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:49.106871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51528 (local address 127.29.229.60:39613)
0811 22:24:49.107096 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.107101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.107208 (+ 107us) server_negotiation.cc:407] Connection header received
0811 22:24:49.107389 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.107398 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.107460 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.107558 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.108311 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.109093 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.110255 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.110951 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.111098 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:49.114565 (+ 3467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:49.114570 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:49.114573 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:49.114600 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:49.117985 (+ 3385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.118669 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.118674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.118677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.118851 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.119350 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.119354 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.119356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.119649 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:49.119812 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:49.120212 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 22:24:49.120687 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":154,"server-negotiator.queue_time_us":61}
I20260811 22:24:49.122016 3908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:49.103467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51274 (local address 127.29.229.61:45157)
0811 22:24:49.104289 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.104297 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.104321 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:24:49.104387 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.104395 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.104517 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.104670 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.106395 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.107553 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.108933 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.109502 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.109690 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:49.115370 (+ 5680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:49.115379 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:49.115388 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:49.115444 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:49.118866 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.119698 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.119706 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.119711 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.119888 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.120325 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.120333 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.120337 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.120745 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:49.121226 (+ 481us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:49.121448 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 22:24:49.121691 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":1223,"threads_started":1}
I20260811 22:24:49.127338 3891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66: Bootstrap starting.
I20260811 22:24:49.127444 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:49.130949 3891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:49.132061 3891 log.cc:824] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:49.130286 3564 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "273c0566d3ab47e6a2a602a07fd83191"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56270
I20260811 22:24:49.135097 3891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66: No bootstrap required, opened a new log
I20260811 22:24:49.142552 3891 raft_consensus.cc:348] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } } peers { permanent_uuid: "4e0e3e24609a4b859260bf4ff0141521" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } } peers { permanent_uuid: "e59f8a152339453d91a55e3ab3b9ff66" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } }
I20260811 22:24:49.142946 3891 raft_consensus.cc:729] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59f8a152339453d91a55e3ab3b9ff66, State: Initialized, Role: FOLLOWER
I20260811 22:24:49.143416 3891 consensus_queue.cc:260] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } } peers { permanent_uuid: "4e0e3e24609a4b859260bf4ff0141521" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } } peers { permanent_uuid: "e59f8a152339453d91a55e3ab3b9ff66" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } }
I20260811 22:24:49.144402 3911 sys_catalog.cc:455] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } } peers { permanent_uuid: "4e0e3e24609a4b859260bf4ff0141521" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } } peers { permanent_uuid: "e59f8a152339453d91a55e3ab3b9ff66" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } } }
I20260811 22:24:49.144830 3911 sys_catalog.cc:458] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:24:49.145856 3891 sys_catalog.cc:564] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:49.148113 3762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } attempt: 5
Aug 11 22:24:49 dist-test-slave-jq4t krb5kdc[3449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487088, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.61@KRBTEST.COM for kudu/127.29.229.60@KRBTEST.COM
W20260811 22:24:49.167240 3935 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:24:49.167409 3935 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:24:49.156563 3774 master_runner.cc:186] Error getting master registration for 127.29.229.62:44339: OK, instance_id { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" instance_seqno: 1786487087796271 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:24:49.168903 3774 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:24:49.171521 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:49.149246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51534 (local address 127.29.229.60:39613)
0811 22:24:49.149464 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.149473 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.149503 (+ 30us) server_negotiation.cc:407] Connection header received
0811 22:24:49.149772 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.149778 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.149874 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.149982 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.150966 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.152531 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.154637 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.155459 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.155593 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:49.160893 (+ 5300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:49.160902 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:49.160909 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:49.161018 (+ 109us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:49.167810 (+ 6792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.169047 (+ 1237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.169053 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.169058 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.169263 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.169734 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.169739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.169742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.170234 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:49.170389 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:49.170869 (+ 480us) server_negotiation.cc:299] Negotiation successful
0811 22:24:49.171124 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:24:49.171540 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:49.149127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51534)
0811 22:24:49.149390 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:49.149415 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:49.149605 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:49.150023 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:49.150028 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:49.150570 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:49.150769 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:49.150779 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.152826 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.152836 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:49.154378 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:49.154402 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.155670 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.155676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:49.155965 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:49.156826 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:49.156851 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:49.160563 (+ 3712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:49.168116 (+ 7553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:49.168122 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:49.168127 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:49.168823 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:49.169396 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:49.169403 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:49.169409 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:49.169604 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:49.170372 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:49.170378 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:49.170565 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 22:24:49.170751 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 22:24:49.171106 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 22:24:49.176914 3762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521: Bootstrap starting.
I20260811 22:24:49.181690 3762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:49.183105 3762 log.cc:824] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:49.186904 3762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521: No bootstrap required, opened a new log
I20260811 22:24:49.198606 3762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } } peers { permanent_uuid: "4e0e3e24609a4b859260bf4ff0141521" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } } peers { permanent_uuid: "e59f8a152339453d91a55e3ab3b9ff66" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } }
I20260811 22:24:49.199076 3762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e3e24609a4b859260bf4ff0141521, State: Initialized, Role: FOLLOWER
I20260811 22:24:49.199635 3762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } } peers { permanent_uuid: "4e0e3e24609a4b859260bf4ff0141521" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } } peers { permanent_uuid: "e59f8a152339453d91a55e3ab3b9ff66" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } }
I20260811 22:24:49.200712 3941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } } peers { permanent_uuid: "4e0e3e24609a4b859260bf4ff0141521" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } } peers { permanent_uuid: "e59f8a152339453d91a55e3ab3b9ff66" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } } }
I20260811 22:24:49.201166 3941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:24:49.202054 3762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:24:49.215613 3963 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:24:49.215853 3963 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:24:49.212612 3628 master_runner.cc:186] Error getting master registration for 127.29.229.62:44339: OK, instance_id { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" instance_seqno: 1786487087796271 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:24:49.225191 3628 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:24:49.229321 3615 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } attempt: 5
Aug 11 22:24:49 dist-test-slave-jq4t krb5kdc[3449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487087, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for kudu/127.29.229.60@KRBTEST.COM
I20260811 22:24:49.247833 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:49.230069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:39613 (local address 127.0.0.1:51550)
0811 22:24:49.230370 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:49.230392 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:49.230518 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:49.230959 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:49.230965 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:49.231502 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:49.231727 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:49.231741 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.233335 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.233341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:49.234896 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:49.234925 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.235973 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.235981 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:49.236199 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:49.237006 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:49.237033 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:49.240054 (+ 3021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:49.244671 (+ 4617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:49.244678 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:49.244682 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:49.245243 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:49.245722 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:49.245727 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:49.245731 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:49.245952 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:49.246661 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:49.246667 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:49.246856 (+ 189us) client_negotiation.cc:769] Sending connection context
0811 22:24:49.247020 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 22:24:49.247359 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260811 22:24:49.249066 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:49.230191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51550 (local address 127.29.229.60:39613)
0811 22:24:49.230415 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.230421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.230446 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:24:49.230663 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.230669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.230784 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.230938 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.231923 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.233147 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.235052 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.235777 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.235923 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:49.240355 (+ 4432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:49.240363 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:49.240369 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:49.240410 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:49.244456 (+ 4046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.245386 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.245392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.245396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.245569 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.246076 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.246083 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.246086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.246462 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:49.246619 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:49.248262 (+ 1643us) server_negotiation.cc:299] Negotiation successful
0811 22:24:49.248597 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:24:49.255343 3615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191: Bootstrap starting.
I20260811 22:24:49.259768 3615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:49.261312 3615 log.cc:824] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:49.266656 3615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191: No bootstrap required, opened a new log
I20260811 22:24:49.275318 3615 raft_consensus.cc:348] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } } peers { permanent_uuid: "4e0e3e24609a4b859260bf4ff0141521" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } } peers { permanent_uuid: "e59f8a152339453d91a55e3ab3b9ff66" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } }
I20260811 22:24:49.275784 3615 raft_consensus.cc:729] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273c0566d3ab47e6a2a602a07fd83191, State: Initialized, Role: FOLLOWER
I20260811 22:24:49.276356 3615 consensus_queue.cc:260] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } } peers { permanent_uuid: "4e0e3e24609a4b859260bf4ff0141521" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } } peers { permanent_uuid: "e59f8a152339453d91a55e3ab3b9ff66" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } }
W20260811 22:24:49.277159 3806 tablet.cc:2369] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:24:49.279366 3969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } } peers { permanent_uuid: "4e0e3e24609a4b859260bf4ff0141521" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } } peers { permanent_uuid: "e59f8a152339453d91a55e3ab3b9ff66" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } } }
I20260811 22:24:49.280159 3969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:24:49.280867 3615 sys_catalog.cc:564] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:49.281097 3564 raft_consensus.cc:482] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 22:24:49.281350 3564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [term 0 FOLLOWER]: Advancing to term 1
W20260811 22:24:49.282392 3518 tablet.cc:2369] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:24:49.285518 3564 raft_consensus.cc:504] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } } peers { permanent_uuid: "4e0e3e24609a4b859260bf4ff0141521" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } } peers { permanent_uuid: "e59f8a152339453d91a55e3ab3b9ff66" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } }
I20260811 22:24:49.288196 3715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "273c0566d3ab47e6a2a602a07fd83191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4e0e3e24609a4b859260bf4ff0141521"
I20260811 22:24:49.288725 3715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:49.288776 3842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "273c0566d3ab47e6a2a602a07fd83191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e59f8a152339453d91a55e3ab3b9ff66"
I20260811 22:24:49.289260 3842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:49.292896 3715 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273c0566d3ab47e6a2a602a07fd83191 in term 1.
I20260811 22:24:49.293202 3842 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273c0566d3ab47e6a2a602a07fd83191 in term 1.
I20260811 22:24:49.293639 3510 leader_election.cc:302] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 273c0566d3ab47e6a2a602a07fd83191, 4e0e3e24609a4b859260bf4ff0141521; no voters:
I20260811 22:24:49.294143 3564 leader_election.cc:288] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [CANDIDATE]: Term 1 election: Requested vote from peers 4e0e3e24609a4b859260bf4ff0141521 (127.29.229.61:45157), e59f8a152339453d91a55e3ab3b9ff66 (127.29.229.60:39613)
I20260811 22:24:49.301331 3969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:49.311342 3969 raft_consensus.cc:686] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [term 1 LEADER]: Becoming Leader. State: Replica: 273c0566d3ab47e6a2a602a07fd83191, State: Running, Role: LEADER
I20260811 22:24:49.312404 3969 consensus_queue.cc:237] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } } peers { permanent_uuid: "4e0e3e24609a4b859260bf4ff0141521" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } } peers { permanent_uuid: "e59f8a152339453d91a55e3ab3b9ff66" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } }
I20260811 22:24:49.324793 3977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 273c0566d3ab47e6a2a602a07fd83191. Latest consensus state: current_term: 1 leader_uuid: "273c0566d3ab47e6a2a602a07fd83191" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } } peers { permanent_uuid: "4e0e3e24609a4b859260bf4ff0141521" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } } peers { permanent_uuid: "e59f8a152339453d91a55e3ab3b9ff66" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } } }
I20260811 22:24:49.325464 3977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:49.331790 3990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:24:49.365398 3990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:49.372157 4010 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 22:24:49.394713 3842 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66 [term 1 FOLLOWER]: Refusing update from remote peer 273c0566d3ab47e6a2a602a07fd83191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260811 22:24:49.397744 3661 tablet.cc:2369] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:24:49.397703 3969 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59f8a152339453d91a55e3ab3b9ff66" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:24:49.407586 3715 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521 [term 1 FOLLOWER]: Refusing update from remote peer 273c0566d3ab47e6a2a602a07fd83191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:24:49.408733 3977 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e3e24609a4b859260bf4ff0141521" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:24:49.425959 3911 sys_catalog.cc:455] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 273c0566d3ab47e6a2a602a07fd83191. Latest consensus state: current_term: 1 leader_uuid: "273c0566d3ab47e6a2a602a07fd83191" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } } peers { permanent_uuid: "4e0e3e24609a4b859260bf4ff0141521" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } } peers { permanent_uuid: "e59f8a152339453d91a55e3ab3b9ff66" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } } }
I20260811 22:24:49.426322 3911 sys_catalog.cc:458] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:24:49.433146 4015 mvcc.cc:205] Tried to move back new op lower bound from 7317451118139338752 to 7317451117841022976. Current Snapshot: MvccSnapshot[applied={T|T < 7317451118139338752}]
I20260811 22:24:49.433734 3911 sys_catalog.cc:455] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "273c0566d3ab47e6a2a602a07fd83191" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } } peers { permanent_uuid: "4e0e3e24609a4b859260bf4ff0141521" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } } peers { permanent_uuid: "e59f8a152339453d91a55e3ab3b9ff66" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } } }
I20260811 22:24:49.434122 3911 sys_catalog.cc:458] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:24:49.439766 3969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "273c0566d3ab47e6a2a602a07fd83191" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } } peers { permanent_uuid: "4e0e3e24609a4b859260bf4ff0141521" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } } peers { permanent_uuid: "e59f8a152339453d91a55e3ab3b9ff66" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } } }
I20260811 22:24:49.440207 3969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:49.442516 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:49.340532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51566 (local address 127.29.229.60:39613)
0811 22:24:49.340752 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.340760 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.366237 (+ 25477us) server_negotiation.cc:407] Connection header received
0811 22:24:49.366523 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.366529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.366651 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.366729 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.381229 (+ 14500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.382590 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.384780 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.385998 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.386108 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:49.412027 (+ 25919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:49.412036 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:49.412042 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:49.412098 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:49.416517 (+ 4419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.422687 (+ 6170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.422694 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.422698 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.422898 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.428185 (+ 5287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.428194 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.428200 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.428753 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:49.429062 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:49.441545 (+ 12483us) server_negotiation.cc:299] Negotiation successful
0811 22:24:49.441974 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:24:49.452250 3990 catalog_manager.cc:1347] Generated new cluster ID: c331019c8dc44c39b2fffcda8e8aee59
I20260811 22:24:49.452385 3990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:49.454514 4016 mvcc.cc:205] Tried to move back new op lower bound from 7317451118139338752 to 7317451117841022976. Current Snapshot: MvccSnapshot[applied={T|T < 7317451118139338752 or (T in {7317451118139338752})}]
I20260811 22:24:49.460310 3906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:49.337843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56276 (local address 127.29.229.62:44339)
0811 22:24:49.338092 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.338100 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.360100 (+ 22000us) server_negotiation.cc:407] Connection header received
0811 22:24:49.360737 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.360744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.360836 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.360912 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.369208 (+ 8296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.370361 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.374470 (+ 4109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.375204 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.375312 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:49.446250 (+ 70938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:49.446273 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:49.446279 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:49.446327 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:49.451482 (+ 5155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.454864 (+ 3382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.454871 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.454875 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.455057 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.455566 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.455571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.455575 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.456017 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:49.456258 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:49.459350 (+ 3092us) server_negotiation.cc:299] Negotiation successful
0811 22:24:49.459689 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:24:49.476284 3941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 273c0566d3ab47e6a2a602a07fd83191. Latest consensus state: current_term: 1 leader_uuid: "273c0566d3ab47e6a2a602a07fd83191" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } } peers { permanent_uuid: "4e0e3e24609a4b859260bf4ff0141521" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } } peers { permanent_uuid: "e59f8a152339453d91a55e3ab3b9ff66" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } } }
I20260811 22:24:49.476713 3941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:24:49.478114 3977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "273c0566d3ab47e6a2a602a07fd83191" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } } peers { permanent_uuid: "4e0e3e24609a4b859260bf4ff0141521" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } } peers { permanent_uuid: "e59f8a152339453d91a55e3ab3b9ff66" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } } }
I20260811 22:24:49.479103 3977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:49.480634 3990 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:49.483155 3990 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:49.484252 4017 mvcc.cc:205] Tried to move back new op lower bound from 7317451118139338752 to 7317451117841022976. Current Snapshot: MvccSnapshot[applied={T|T < 7317451118139338752}]
I20260811 22:24:49.489919 3941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "273c0566d3ab47e6a2a602a07fd83191" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "273c0566d3ab47e6a2a602a07fd83191" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 44339 } } peers { permanent_uuid: "4e0e3e24609a4b859260bf4ff0141521" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 45157 } } peers { permanent_uuid: "e59f8a152339453d91a55e3ab3b9ff66" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 39613 } } }
I20260811 22:24:49.490204 3941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:24:49.494802 3990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 273c0566d3ab47e6a2a602a07fd83191: Generated new TSK 0
I20260811 22:24:49.495940 3990 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:24:49.502429 3908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:49.373067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51280 (local address 127.29.229.61:45157)
0811 22:24:49.373274 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.373280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.428056 (+ 54776us) server_negotiation.cc:407] Connection header received
0811 22:24:49.445526 (+ 17470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.445533 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.445638 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.445742 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.456924 (+ 11182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.458592 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.474633 (+ 16041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:49.475331 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:49.475448 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:49.480807 (+ 5359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:49.480815 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:49.480820 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:49.480886 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:49.485388 (+ 4502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.486456 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.486462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.486468 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.486661 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:49.487152 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:49.487157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:49.487162 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:49.487601 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:49.487879 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:49.501606 (+ 13727us) server_negotiation.cc:299] Negotiation successful
0811 22:24:49.501961 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 22:24:49.688980 3906 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:49.663012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56290 (local address 127.29.229.62:44339)
0811 22:24:49.663219 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.663226 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.686546 (+ 23320us) server_negotiation.cc:407] Connection header received
0811 22:24:49.687140 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.687148 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.687241 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.687326 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.688659 (+ 1333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56290: BlockingRecv error: recv got EOF from 127.0.0.1:56290 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 22:24:49.692950 3894 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:49.670903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51572 (local address 127.29.229.60:39613)
0811 22:24:49.671114 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.671120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.690103 (+ 18983us) server_negotiation.cc:407] Connection header received
0811 22:24:49.690950 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.690956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.691051 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.691129 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.692672 (+ 1543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51572: BlockingRecv error: recv got EOF from 127.0.0.1:51572 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260811 22:24:49.696655 3908 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:49.694044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51282 (local address 127.29.229.61:45157)
0811 22:24:49.694288 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.694294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.694954 (+ 660us) server_negotiation.cc:407] Connection header received
0811 22:24:49.695219 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.695225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.695307 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.695407 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.696369 (+ 962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51282: BlockingRecv error: recv got EOF from 127.0.0.1:51282 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260811 22:24:49.738677 3908 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:49.725255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51292 (local address 127.29.229.61:45157)
0811 22:24:49.725480 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.725489 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.736177 (+ 10688us) server_negotiation.cc:407] Connection header received
0811 22:24:49.736872 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.736881 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.737066 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.737163 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.738395 (+ 1232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51292: BlockingRecv error: recv got EOF from 127.0.0.1:51292 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 22:24:49.792328 3908 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:49.777215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51296 (local address 127.29.229.61:45157)
0811 22:24:49.777440 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.777447 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.789135 (+ 11688us) server_negotiation.cc:407] Connection header received
0811 22:24:49.790221 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.790229 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.790346 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.790459 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.791860 (+ 1401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51296: BlockingRecv error: recv got EOF from 127.0.0.1:51296 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 22:24:49.839895 3908 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:49.826680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51300 (local address 127.29.229.61:45157)
0811 22:24:49.826892 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.826899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.837592 (+ 10693us) server_negotiation.cc:407] Connection header received
0811 22:24:49.838174 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.838181 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.838293 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.838397 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.839601 (+ 1204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51300: BlockingRecv error: recv got EOF from 127.0.0.1:51300 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260811 22:24:49.893740 3908 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:49.879926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51314 (local address 127.29.229.61:45157)
0811 22:24:49.880151 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.880159 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.891467 (+ 11308us) server_negotiation.cc:407] Connection header received
0811 22:24:49.891787 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.891795 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.891902 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.891980 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.893458 (+ 1478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51314: BlockingRecv error: recv got EOF from 127.0.0.1:51314 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 22:24:49.909302 3908 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:49.906924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51328 (local address 127.29.229.61:45157)
0811 22:24:49.907142 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.907150 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.907703 (+ 553us) server_negotiation.cc:407] Connection header received
0811 22:24:49.907877 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.907884 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.908011 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.908138 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.909045 (+ 907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51328: BlockingRecv error: recv got EOF from 127.0.0.1:51328 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 22:24:49.933425 3908 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:49.931261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51342 (local address 127.29.229.61:45157)
0811 22:24:49.931482 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.931488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.931834 (+ 346us) server_negotiation.cc:407] Connection header received
0811 22:24:49.932010 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.932017 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.932120 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.932246 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.933173 (+ 927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51342: BlockingRecv error: recv got EOF from 127.0.0.1:51342 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 22:24:49.971288 3908 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:49.969369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51358 (local address 127.29.229.61:45157)
0811 22:24:49.969629 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:49.969636 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:49.969887 (+ 251us) server_negotiation.cc:407] Connection header received
0811 22:24:49.970059 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:49.970065 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:49.970158 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:49.970277 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:49.971028 (+ 751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51358: BlockingRecv error: recv got EOF from 127.0.0.1:51358 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
W20260811 22:24:50.041757 3908 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:50.039001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51362 (local address 127.29.229.61:45157)
0811 22:24:50.039214 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:50.039219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:50.039560 (+ 341us) server_negotiation.cc:407] Connection header received
0811 22:24:50.039873 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:50.039880 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:50.040000 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:50.040145 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:50.041266 (+ 1121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51362: BlockingRecv error: recv got EOF from 127.0.0.1:51362 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 22:24:50.169749 3935 catalog_manager.cc:1259] Loaded cluster ID: c331019c8dc44c39b2fffcda8e8aee59
I20260811 22:24:50.169919 3935 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66: loading cluster ID for follower catalog manager: success
I20260811 22:24:50.172770 3935 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66: acquiring CA information for follower catalog manager: success
I20260811 22:24:50.174929 3935 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e59f8a152339453d91a55e3ab3b9ff66: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 22:24:50.175396 3908 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:50.173301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51368 (local address 127.29.229.61:45157)
0811 22:24:50.173526 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:50.173535 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:50.173909 (+ 374us) server_negotiation.cc:407] Connection header received
0811 22:24:50.174164 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:50.174169 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:50.174261 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:50.174373 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:50.175132 (+ 759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51368: BlockingRecv error: recv got EOF from 127.0.0.1:51368 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:24:50.218030 3963 catalog_manager.cc:1259] Loaded cluster ID: c331019c8dc44c39b2fffcda8e8aee59
I20260811 22:24:50.218142 3963 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521: loading cluster ID for follower catalog manager: success
I20260811 22:24:50.220785 3963 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521: acquiring CA information for follower catalog manager: success
I20260811 22:24:50.222563 3963 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4e0e3e24609a4b859260bf4ff0141521: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:24:50.441705 3908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:50.435809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51380 (local address 127.29.229.61:45157)
0811 22:24:50.436059 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:50.436065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:50.436294 (+ 229us) server_negotiation.cc:407] Connection header received
0811 22:24:50.436580 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:50.436584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:50.436752 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:50.436893 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:24:50.437406 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:50.438124 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:50.439497 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:50.439895 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:50.440032 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:50.440553 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:24:50.441081 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:24:50.441244 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:50.441352 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 22:24:50.441439 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 22:24:50.456334 4158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:50.449497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51578 (local address 127.29.229.60:39613)
0811 22:24:50.450025 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:50.450030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:50.450439 (+ 409us) server_negotiation.cc:407] Connection header received
0811 22:24:50.450715 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:50.450720 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:50.450915 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:50.451090 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:24:50.451538 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:50.452274 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:50.453728 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:50.454183 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:50.454288 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:50.454827 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:24:50.455452 (+ 625us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:24:50.455575 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:50.455768 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 22:24:50.455849 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":286,"threads_started":1}
I20260811 22:24:50.473591 3908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:50.466301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51382 (local address 127.29.229.61:45157)
0811 22:24:50.466510 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:50.466515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:50.467084 (+ 569us) server_negotiation.cc:407] Connection header received
0811 22:24:50.467403 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:50.467410 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:50.467511 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:50.467613 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:24:50.468247 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:50.469245 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:50.471511 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:50.472029 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:50.472176 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:50.472687 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:24:50.472793 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:24:50.473005 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:50.473157 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 22:24:50.473260 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:24:50.474010 4158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:50.464909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51580 (local address 127.29.229.60:39613)
0811 22:24:50.465191 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:50.465201 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:50.466241 (+ 1040us) server_negotiation.cc:407] Connection header received
0811 22:24:50.466552 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:50.466560 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:50.466671 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:50.466749 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:24:50.467797 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:50.468733 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:50.471017 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:50.471719 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:50.471837 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:50.473012 (+ 1175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:24:50.473171 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:24:50.473264 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:50.473601 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 22:24:50.473694 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 22:24:50.474555 4167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:50.464649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56292 (local address 127.29.229.62:44339)
0811 22:24:50.465627 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:50.465636 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:50.466353 (+ 717us) server_negotiation.cc:407] Connection header received
0811 22:24:50.466522 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:50.466531 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:50.466793 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:50.467023 (+ 230us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:24:50.467801 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:50.468879 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:50.470999 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:50.471397 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:50.471542 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:50.472520 (+ 978us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:24:50.473050 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:24:50.473152 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:50.473713 (+ 561us) server_negotiation.cc:299] Negotiation successful
0811 22:24:50.473818 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":822,"thread_start_us":682,"threads_started":1}
I20260811 22:24:50.479744 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 3476
I20260811 22:24:50.481333 3607 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:24:50.597137 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 3476
W20260811 22:24:50.631341 3659 connection.cc:535] server connection from 127.0.0.1:51254 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 22:24:50.631340 3800 connection.cc:535] server connection from 127.0.0.1:51550 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:24:50.631824 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 3628
I20260811 22:24:50.633183 3755 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:24:50.745163 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 3628
I20260811 22:24:50.773583 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 3774
I20260811 22:24:50.774775 3882 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:24:50.888090 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 3774
2026-08-11T22:24:50Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5453 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:24:51 dist-test-slave-jq4t krb5kdc[4183](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:24:51 dist-test-slave-jq4t krb5kdc[4183](info): set up 2 sockets
Aug 11 22:24:51 dist-test-slave-jq4t krb5kdc[4183](info): commencing operation
krb5kdc: starting...
W20260811 22:24:52.315436 30612 mini_kdc.cc:121] Time spent starting KDC: real 1.377s user 0.004s sys 0.061s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/test-user.keytab.
Aug 11 22:24:52 dist-test-slave-jq4t krb5kdc[4183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:24:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:24:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.229.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
Entry for principal kudu/127.29.229.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab.
I20260811 22:24:52.713241 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:40533
--webserver_interface=127.29.229.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:42491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:40533,127.29.229.61:46837,127.29.229.60:41389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:24:53.080217 4209 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:24:53.080662 4209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:53.080799 4209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:53.080941 4209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:53.088268 4209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:53.088405 4209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:53.088541 4209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:53.088639 4209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:53.088733 4209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:53.098807 4209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:42491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:40533,127.29.229.61:46837,127.29.229.60:41389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.62.keytab
--principal=kudu/127.29.229.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.62:40533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.229.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:53.100294 4209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:53.101836 4209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:53.116627 4216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:53.116748 4224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:53.116709 4219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:53.117718 4209 server_base.cc:1034] running on GCE node
I20260811 22:24:53.118676 4209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:53.120249 4209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:53.121562 4209 hybrid_clock.cc:648] HybridClock initialized: now 1786487093121464 us; error 62 us; skew 500 ppm
I20260811 22:24:53.121958 4209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:53 dist-test-slave-jq4t krb5kdc[4183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487093, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:53.130436 4209 init.cc:377] Logged in from keytab as kudu/127.29.229.62@KRBTEST.COM (short username kudu)
I20260811 22:24:53.132216 4209 webserver.cc:460] Webserver started at http://127.29.229.62:42995/ using document root <none> and password file <none>
I20260811 22:24:53.133064 4209 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:53.133423 4209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:53.133958 4209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:53.137902 4209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-0/data/instance:
uuid: "79ca578857d54c43862e387f55a51788"
format_stamp: "Formatted at 2026-08-11 22:24:53 on dist-test-slave-jq4t"
server_key: "e6f9a790d8b27a1c31dcf79628e06526"
server_key_iv: "45ca8cbb36734c926d0d90b6e18aa191"
server_key_version: "encryptionkey@0"
I20260811 22:24:53.138839 4209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-0/wal/instance:
uuid: "79ca578857d54c43862e387f55a51788"
format_stamp: "Formatted at 2026-08-11 22:24:53 on dist-test-slave-jq4t"
server_key: "e6f9a790d8b27a1c31dcf79628e06526"
server_key_iv: "45ca8cbb36734c926d0d90b6e18aa191"
server_key_version: "encryptionkey@0"
I20260811 22:24:53.145678 4209 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 22:24:53.149492 4239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:53.150897 4209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:24:53.151099 4209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-0/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-0/wal
uuid: "79ca578857d54c43862e387f55a51788"
format_stamp: "Formatted at 2026-08-11 22:24:53 on dist-test-slave-jq4t"
server_key: "e6f9a790d8b27a1c31dcf79628e06526"
server_key_iv: "45ca8cbb36734c926d0d90b6e18aa191"
server_key_version: "encryptionkey@0"
I20260811 22:24:53.151355 4209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:53.171657 4209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:53.176568 4209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:53.176906 4209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:53.197614 4209 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.62:40533
I20260811 22:24:53.197642 4342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.62:40533 every 8 connection(s)
I20260811 22:24:53.199067 4209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-0/data/info.pb
I20260811 22:24:53.203071 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 4209
I20260811 22:24:53.203293 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-0/wal/instance
I20260811 22:24:53.203894 30612 external_mini_cluster.cc:1428] Setting key ccd38dbaf29850361bf6ddbc02ca4f0c
I20260811 22:24:53.204918 4343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:53.216997 4343 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:24:53 dist-test-slave-jq4t krb5kdc[4183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487093, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.229.61@KRBTEST.COM; defaulting to no policy
I20260811 22:24:53.243316 4353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.222284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57416 (local address 127.29.229.62:40533)
0811 22:24:53.224068 (+ 1784us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:53.224076 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:53.224138 (+ 62us) server_negotiation.cc:407] Connection header received
0811 22:24:53.225850 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:53.225856 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:53.226102 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:53.226320 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:53.227661 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:53.229127 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:53.230751 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:53.231335 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:53.231561 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:53.235563 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:53.235572 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:53.235585 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:53.235615 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:53.238616 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:53.239332 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:53.239348 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:53.239350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:53.239502 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:53.239913 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:53.239920 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:53.239923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:53.240437 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:53.240722 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:53.241276 (+ 554us) server_negotiation.cc:299] Negotiation successful
0811 22:24:53.241664 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":1563,"thread_start_us":1341,"threads_started":1}
I20260811 22:24:53.243330 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.222284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:40533 (local address 127.0.0.1:57416)
0811 22:24:53.223232 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.223287 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:53.225464 (+ 2177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:53.226337 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:53.226344 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:53.227082 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:53.227428 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:53.227451 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:53.229323 (+ 1872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:53.229330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:53.230583 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:53.230596 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:53.231590 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:53.231597 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:53.231777 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:53.232585 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:53.232610 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:53.235310 (+ 2700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:53.238756 (+ 3446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:53.238769 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:53.238783 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:53.239195 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:53.239619 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:53.239626 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:53.239631 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:53.239804 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:53.240641 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:53.240654 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:53.240894 (+ 240us) client_negotiation.cc:769] Sending connection context
0811 22:24:53.241156 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 22:24:53.241790 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"mutex_wait_us":148,"thread_start_us":420,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.61@KRBTEST.COM" created.
I20260811 22:24:53.246640 4343 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:24:53.247903 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.247469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:46837 (local address 127.0.0.1:54802)
0811 22:24:53.247649 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.247762 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:46837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 22:24:53.248208 4243 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.229.61:46837: connect: Connection refused (error 111)
I20260811 22:24:53.250041 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.249594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:46837 (local address 127.0.0.1:54806)
0811 22:24:53.249848 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.249925 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:46837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
W20260811 22:24:53.250248 4343 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.61:46837: Network error: Client connection negotiation failed: client connection to 127.29.229.61:46837: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.61.keytab.
Entry for principal kudu/127.29.229.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.61.keytab.
I20260811 22:24:53.292893 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.61:46837
--webserver_interface=127.29.229.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.61.keytab
--principal=kudu/127.29.229.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:42491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:40533,127.29.229.61:46837,127.29.229.60:41389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:24:53.299625 4343 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } attempt: 1
I20260811 22:24:53.300849 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.300352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:46837 (local address 127.0.0.1:54822)
0811 22:24:53.300578 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.300701 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:46837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 22:24:53.302066 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.301635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:46837 (local address 127.0.0.1:54828)
0811 22:24:53.301817 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.301926 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:46837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 22:24:53.302325 4343 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.61:46837: Network error: Client connection negotiation failed: client connection to 127.29.229.61:46837: connect: Connection refused (error 111)
I20260811 22:24:53.370625 4343 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } attempt: 2
I20260811 22:24:53.372012 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.371413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:46837 (local address 127.0.0.1:54840)
0811 22:24:53.371693 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.371820 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:46837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 22:24:53.373265 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.372807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:46837 (local address 127.0.0.1:54842)
0811 22:24:53.372971 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.373077 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:46837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 22:24:53.373543 4343 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.61:46837: Network error: Client connection negotiation failed: client connection to 127.29.229.61:46837: connect: Connection refused (error 111)
I20260811 22:24:53.464846 4343 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } attempt: 3
I20260811 22:24:53.466065 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.465556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:46837 (local address 127.0.0.1:54858)
0811 22:24:53.465786 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.465900 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:46837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 22:24:53.467207 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.466789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:46837 (local address 127.0.0.1:54860)
0811 22:24:53.466957 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.467054 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:46837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 22:24:53.467424 4343 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.61:46837: Network error: Client connection negotiation failed: client connection to 127.29.229.61:46837: connect: Connection refused (error 111)
I20260811 22:24:53.610713 4343 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } attempt: 4
I20260811 22:24:53.612121 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.611463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:46837 (local address 127.0.0.1:54870)
0811 22:24:53.611812 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.611928 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:46837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
I20260811 22:24:53.613392 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.612931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:46837 (local address 127.0.0.1:54876)
0811 22:24:53.613103 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.613213 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.61:46837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 22:24:53.613659 4343 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.61:46837: Network error: Client connection negotiation failed: client connection to 127.29.229.61:46837: connect: Connection refused (error 111)
W20260811 22:24:53.671811 4358 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:24:53.672199 4358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:53.672297 4358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:53.672466 4358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:53.680049 4358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:53.680156 4358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:53.680218 4358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:53.680303 4358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:53.680436 4358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:53.690923 4358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:42491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:40533,127.29.229.61:46837,127.29.229.60:41389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.61.keytab
--principal=kudu/127.29.229.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.61:46837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.229.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:53.692382 4358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:53.693811 4358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:53.706259 4370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:53.706355 4372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:53.706322 4374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:53.707394 4358 server_base.cc:1034] running on GCE node
I20260811 22:24:53.708293 4358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:53.709826 4358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:53.711100 4358 hybrid_clock.cc:648] HybridClock initialized: now 1786487093711040 us; error 32 us; skew 500 ppm
I20260811 22:24:53.711479 4358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:53 dist-test-slave-jq4t krb5kdc[4183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487093, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:53.715075 4358 init.cc:377] Logged in from keytab as kudu/127.29.229.61@KRBTEST.COM (short username kudu)
I20260811 22:24:53.716835 4358 webserver.cc:460] Webserver started at http://127.29.229.61:38399/ using document root <none> and password file <none>
I20260811 22:24:53.717587 4358 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:53.717738 4358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:53.718097 4358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:53.721405 4358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-1/data/instance:
uuid: "df0c35b4e2de494280e4e2164feb9fae"
format_stamp: "Formatted at 2026-08-11 22:24:53 on dist-test-slave-jq4t"
server_key: "d39fee63e402439d623f962d9889ae9b"
server_key_iv: "49e9964f7847adcc7e746ec824035db6"
server_key_version: "encryptionkey@0"
I20260811 22:24:53.723173 4358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-1/wal/instance:
uuid: "df0c35b4e2de494280e4e2164feb9fae"
format_stamp: "Formatted at 2026-08-11 22:24:53 on dist-test-slave-jq4t"
server_key: "d39fee63e402439d623f962d9889ae9b"
server_key_iv: "49e9964f7847adcc7e746ec824035db6"
server_key_version: "encryptionkey@0"
I20260811 22:24:53.729205 4358 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260811 22:24:53.733289 4382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:53.734848 4358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:24:53.735060 4358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-1/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-1/wal
uuid: "df0c35b4e2de494280e4e2164feb9fae"
format_stamp: "Formatted at 2026-08-11 22:24:53 on dist-test-slave-jq4t"
server_key: "d39fee63e402439d623f962d9889ae9b"
server_key_iv: "49e9964f7847adcc7e746ec824035db6"
server_key_version: "encryptionkey@0"
I20260811 22:24:53.735296 4358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:53.751334 4358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:53.758108 4358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:53.758468 4358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:53.777901 4358 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.61:46837
I20260811 22:24:53.777909 4473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.61:46837 every 8 connection(s)
I20260811 22:24:53.779366 4358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-1/data/info.pb
I20260811 22:24:53.783427 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 4358
I20260811 22:24:53.783488 4474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:53.783788 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-1/wal/instance
I20260811 22:24:53.784405 30612 external_mini_cluster.cc:1428] Setting key f9b5c449ce2869b74815bc07b2a384b1
I20260811 22:24:53.790560 4474 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:24:53 dist-test-slave-jq4t krb5kdc[4183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487093, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.61@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:24:53.815564 4478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.794502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57432 (local address 127.29.229.62:40533)
0811 22:24:53.795235 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:53.795241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:53.796167 (+ 926us) server_negotiation.cc:407] Connection header received
0811 22:24:53.798309 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:53.798317 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:53.798421 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:53.798551 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:53.800275 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:53.801084 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:53.802939 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:53.803681 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:53.803856 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:53.807587 (+ 3731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:53.807596 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:53.807604 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:53.807667 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:53.812058 (+ 4391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:53.813009 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:53.813015 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:53.813019 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:53.813159 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:53.813648 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:53.813654 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:53.813658 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:53.813958 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:53.814137 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:53.814936 (+ 799us) server_negotiation.cc:299] Negotiation successful
0811 22:24:53.815170 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":419,"threads_started":1}
I20260811 22:24:53.816640 4479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.794998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:40533 (local address 127.0.0.1:57432)
0811 22:24:53.795955 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.795998 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:53.797970 (+ 1972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:53.798829 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:53.798839 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:53.799418 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:53.800030 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:53.800049 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:53.801264 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:53.801268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:53.802540 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:53.802553 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:53.803964 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:53.803970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:53.804192 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:53.805006 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:53.805036 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:53.807304 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:53.812262 (+ 4958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:53.812274 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:53.812294 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:53.812875 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:53.813290 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:53.813295 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:53.813305 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:53.813518 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:53.814098 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:53.814107 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:53.814508 (+ 401us) client_negotiation.cc:769] Sending connection context
0811 22:24:53.814790 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 22:24:53.815348 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":400,"threads_started":1}
I20260811 22:24:53.818609 4474 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.229.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.229.60@KRBTEST.COM" created.
Aug 11 22:24:53 dist-test-slave-jq4t krb5kdc[4183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487093, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.61@KRBTEST.COM for kudu/127.29.229.61@KRBTEST.COM
I20260811 22:24:53.837551 4486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.819717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54882 (local address 127.29.229.61:46837)
0811 22:24:53.820464 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:53.820515 (+ 51us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:53.820557 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:24:53.820605 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:53.820611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:53.820801 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:53.820923 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:53.821998 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:53.823153 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:53.825356 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:53.826367 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:53.826543 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:53.830888 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:53.830902 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:53.830917 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:53.830950 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:53.834135 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:53.835054 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:53.835065 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:53.835071 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:53.835225 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:53.835693 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:53.835699 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:53.835705 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:53.836187 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:53.836387 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:53.836902 (+ 515us) server_negotiation.cc:299] Negotiation successful
0811 22:24:53.837231 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":595,"thread_start_us":442,"threads_started":1}
I20260811 22:24:53.837581 4479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.819453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:46837 (local address 127.0.0.1:54882)
0811 22:24:53.819732 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.819758 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:53.819903 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:53.820971 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:53.820985 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:53.821562 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:53.821792 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:53.821807 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:53.823309 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:53.823317 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:53.825166 (+ 1849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:53.825188 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:53.826588 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:53.826593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:53.826740 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:53.827751 (+ 1011us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:53.827770 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:53.830602 (+ 2832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:53.834306 (+ 3704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:53.834314 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:53.834321 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:53.834878 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:53.835380 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:53.835387 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:53.835393 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:53.835566 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:53.836340 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:53.836348 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:53.836563 (+ 215us) client_negotiation.cc:769] Sending connection context
0811 22:24:53.836728 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 22:24:53.837087 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 22:24:53.840590 4474 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:24:53.842041 4479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.841407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:52904)
0811 22:24:53.841643 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.841807 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260811 22:24:53.842365 4386 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
I20260811 22:24:53.844424 4479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.843951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:52910)
0811 22:24:53.844151 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.844260 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 22:24:53.844704 4474 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:41389: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.229.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.60.keytab.
Entry for principal kudu/127.29.229.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.60.keytab.
I20260811 22:24:53.876894 30612 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
/tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.60:41389
--webserver_interface=127.29.229.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.60.keytab
--principal=kudu/127.29.229.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.229.20:42491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.229.62:40533,127.29.229.61:46837,127.29.229.60:41389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:24:53.894080 4474 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } attempt: 1
I20260811 22:24:53.895367 4479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.894903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:52926)
0811 22:24:53.895107 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.895214 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 22:24:53.896600 4479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.896140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:52940)
0811 22:24:53.896324 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.896402 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 22:24:53.896874 4474 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:41389: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
I20260811 22:24:53.912928 4343 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } attempt: 5
Aug 11 22:24:53 dist-test-slave-jq4t krb5kdc[4183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487093, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for kudu/127.29.229.61@KRBTEST.COM
I20260811 22:24:53.929476 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.913818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:46837 (local address 127.0.0.1:54892)
0811 22:24:53.914105 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.914145 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:53.914329 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:53.914890 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:53.914893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:53.915298 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:53.915574 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:53.915585 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:53.917231 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:53.917235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:53.918136 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:53.918148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:53.919186 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:53.919191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:53.919391 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:53.920113 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:53.920132 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:53.922655 (+ 2523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:53.925936 (+ 3281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:53.925941 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:53.925945 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:53.926521 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:53.927324 (+ 803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:53.927329 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:53.927332 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:53.927532 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:53.928319 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:53.928326 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:53.928572 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 22:24:53.928743 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 22:24:53.929104 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 22:24:53.929597 4486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.914065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54892 (local address 127.29.229.61:46837)
0811 22:24:53.914382 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:53.914391 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:53.914418 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:24:53.914517 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:53.914526 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:53.914651 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:53.914783 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:53.915755 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:53.917031 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:53.918291 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:53.919003 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:53.919134 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:53.922915 (+ 3781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:53.922920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:53.922923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:53.922963 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:53.925771 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:53.926795 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:53.926803 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:53.926806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:53.927110 (+ 304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:53.927714 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:53.927719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:53.927721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:53.928163 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:53.928332 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:53.929022 (+ 690us) server_negotiation.cc:299] Negotiation successful
0811 22:24:53.929241 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 22:24:53.930891 4343 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:24:53.932767 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.932177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:52952)
0811 22:24:53.932421 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.932610 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260811 22:24:53.933933 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.933580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:52962)
0811 22:24:53.933731 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.933826 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 22:24:53.934149 4343 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:41389: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
I20260811 22:24:53.965135 4474 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } attempt: 2
I20260811 22:24:53.966435 4479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.965940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:52968)
0811 22:24:53.966172 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.966267 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 22:24:53.967494 4479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.967133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:52976)
0811 22:24:53.967297 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.967377 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 22:24:53.967721 4474 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:41389: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
I20260811 22:24:53.985409 4343 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } attempt: 1
I20260811 22:24:53.986713 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.986232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:52988)
0811 22:24:53.986458 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.986544 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 22:24:53.987864 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:53.987491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:52996)
0811 22:24:53.987670 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:53.987750 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 22:24:53.988082 4343 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:41389: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
I20260811 22:24:54.056354 4343 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } attempt: 2
I20260811 22:24:54.057566 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.057058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:53006)
0811 22:24:54.057290 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:54.057399 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 22:24:54.058646 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.058272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:53016)
0811 22:24:54.058473 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:54.058540 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 22:24:54.058887 4343 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:41389: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
I20260811 22:24:54.058928 4474 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } attempt: 3
I20260811 22:24:54.059906 4479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.059471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:53020)
0811 22:24:54.059670 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:54.059761 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 22:24:54.061003 4479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.060571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:53034)
0811 22:24:54.060796 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:54.060890 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260811 22:24:54.061241 4474 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:41389: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
I20260811 22:24:54.165160 4343 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } attempt: 3
I20260811 22:24:54.166349 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.165872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:53036)
0811 22:24:54.166074 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:54.166171 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 22:24:54.167701 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.167266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:53052)
0811 22:24:54.167512 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:54.167586 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260811 22:24:54.167973 4343 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:41389: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
I20260811 22:24:54.204502 4474 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } attempt: 4
I20260811 22:24:54.206051 4479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.205258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:53062)
0811 22:24:54.205671 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:54.205816 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260811 22:24:54.207430 4479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.207012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:53078)
0811 22:24:54.207198 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:54.207294 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 22:24:54.207695 4474 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:41389: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
W20260811 22:24:54.257262 4491 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:24:54.257679 4491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:24:54.257809 4491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:24:54.257966 4491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:24:54.265511 4491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:24:54.265643 4491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:24:54.265774 4491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:24:54.265866 4491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:24:54.265971 4491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:24:54.276214 4491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.229.20:42491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.229.62:40533,127.29.229.61:46837,127.29.229.60:41389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/krb5kdc/kudu_127.29.229.60.keytab
--principal=kudu/127.29.229.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.229.60:41389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.229.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:24:54.277683 4491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:24:54.279052 4491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:24:54.291519 4498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:54.291595 4500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:24:54.291589 4505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:24:54.293147 4491 server_base.cc:1034] running on GCE node
I20260811 22:24:54.294064 4491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:24:54.295528 4491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:24:54.296751 4491 hybrid_clock.cc:648] HybridClock initialized: now 1786487094296658 us; error 56 us; skew 500 ppm
I20260811 22:24:54.297144 4491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:24:54 dist-test-slave-jq4t krb5kdc[4183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487094, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:24:54.300762 4491 init.cc:377] Logged in from keytab as kudu/127.29.229.60@KRBTEST.COM (short username kudu)
I20260811 22:24:54.302767 4491 webserver.cc:460] Webserver started at http://127.29.229.60:36789/ using document root <none> and password file <none>
I20260811 22:24:54.303557 4491 fs_manager.cc:359] Metadata directory not provided
I20260811 22:24:54.303705 4491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:24:54.304064 4491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:24:54.307380 4491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-2/data/instance:
uuid: "b8a9366146cd411180b032bd7d8dea20"
format_stamp: "Formatted at 2026-08-11 22:24:54 on dist-test-slave-jq4t"
server_key: "f6e307f6c87b3554e73fe6b90d8e48e5"
server_key_iv: "acfaa98acf875f1bca4d93f1c7cf0f16"
server_key_version: "encryptionkey@0"
I20260811 22:24:54.308230 4491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-2/wal/instance:
uuid: "b8a9366146cd411180b032bd7d8dea20"
format_stamp: "Formatted at 2026-08-11 22:24:54 on dist-test-slave-jq4t"
server_key: "f6e307f6c87b3554e73fe6b90d8e48e5"
server_key_iv: "acfaa98acf875f1bca4d93f1c7cf0f16"
server_key_version: "encryptionkey@0"
I20260811 22:24:54.314657 4491 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 22:24:54.319077 4522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:24:54.320852 4491 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:24:54.321080 4491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-2/data,/tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-2/wal
uuid: "b8a9366146cd411180b032bd7d8dea20"
format_stamp: "Formatted at 2026-08-11 22:24:54 on dist-test-slave-jq4t"
server_key: "f6e307f6c87b3554e73fe6b90d8e48e5"
server_key_iv: "acfaa98acf875f1bca4d93f1c7cf0f16"
server_key_version: "encryptionkey@0"
I20260811 22:24:54.321345 4491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:24:54.339679 4491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:24:54.344740 4491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:24:54.345082 4491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:24:54.345254 4343 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } attempt: 4
I20260811 22:24:54.346486 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.345919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:53080)
0811 22:24:54.346181 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:54.346299 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 22:24:54.347652 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.347215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:53088)
0811 22:24:54.347390 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:54.347487 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 22:24:54.347903 4343 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.229.60:41389: Network error: Client connection negotiation failed: client connection to 127.29.229.60:41389: connect: Connection refused (error 111)
I20260811 22:24:54.366952 4491 rpc_server.cc:307] RPC server started. Bound to: 127.29.229.60:41389
I20260811 22:24:54.366972 4609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.229.60:41389 every 8 connection(s)
I20260811 22:24:54.368419 4491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-2/data/info.pb
I20260811 22:24:54.372466 4613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:24:54.378482 30612 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu as pid 4491
I20260811 22:24:54.378696 30612 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskD_Yjlj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486685119861-30612-0/minicluster-data/master-2/wal/instance
I20260811 22:24:54.379292 30612 external_mini_cluster.cc:1428] Setting key dcc92ddce2511f7ecd15cc9327a462cf
I20260811 22:24:54.379982 4613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:24:54 dist-test-slave-jq4t krb5kdc[4183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487094, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.60@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
Aug 11 22:24:54 dist-test-slave-jq4t krb5kdc[4183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.60@KRBTEST.COM
I20260811 22:24:54.408041 4621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.383372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57448 (local address 127.29.229.62:40533)
0811 22:24:54.385088 (+ 1716us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:54.385095 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:54.385121 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:24:54.385642 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:54.385648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:54.385750 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:54.385886 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:54.388475 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.389973 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.392424 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.393310 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.393507 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:54.399647 (+ 6140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:54.399654 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:54.399658 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:54.399740 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:54.403994 (+ 4254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.405118 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.405122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.405124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.405269 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.405802 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.405807 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.405809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.406205 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:54.406377 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:54.407302 (+ 925us) server_negotiation.cc:299] Negotiation successful
0811 22:24:54.407570 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1570,"thread_start_us":1392,"threads_started":1}
I20260811 22:24:54.409061 4617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.381396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53096 (local address 127.29.229.60:41389)
0811 22:24:54.382235 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:54.382246 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:54.382288 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:24:54.385134 (+ 2846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:54.385168 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:54.385475 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:54.385739 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:54.388531 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.390481 (+ 1950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.392407 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.393683 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.393942 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:54.399906 (+ 5964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:54.399919 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:54.399939 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:54.399985 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:54.404343 (+ 4358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.405380 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.405389 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.405392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.405572 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.405999 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.406003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.406006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.406680 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:54.406943 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:54.407387 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 22:24:54.407832 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":142,"server-negotiator.queue_time_us":631,"spinlock_wait_cycles":170112,"thread_start_us":381,"threads_started":1}
I20260811 22:24:54.409101 4619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.383262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.62:40533 (local address 127.0.0.1:57448)
0811 22:24:54.384768 (+ 1506us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:54.384854 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:54.385325 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:54.386022 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:54.386032 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:54.387268 (+ 1236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:54.388138 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:54.388175 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.390211 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.390216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:54.392200 (+ 1984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:54.392217 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.393643 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.393648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:54.393985 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:54.395119 (+ 1134us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:54.395161 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:54.399330 (+ 4169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:54.404228 (+ 4898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:54.404239 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:54.404250 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:54.404942 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:54.405441 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:54.405445 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:54.405447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:54.405672 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:54.406356 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:54.406363 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:54.406699 (+ 336us) client_negotiation.cc:769] Sending connection context
0811 22:24:54.407070 (+ 371us) client_negotiation.cc:240] Negotiation successful
0811 22:24:54.407655 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":889,"mutex_wait_us":71,"spinlock_wait_cycles":1408,"thread_start_us":566,"threads_started":1}
I20260811 22:24:54.410900 4613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:24:54 dist-test-slave-jq4t krb5kdc[4183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487094, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.60@KRBTEST.COM for kudu/127.29.229.61@KRBTEST.COM
Aug 11 22:24:54 dist-test-slave-jq4t krb5kdc[4183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.62@KRBTEST.COM
I20260811 22:24:54.427433 4619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.412165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.61:46837 (local address 127.0.0.1:54904)
0811 22:24:54.412401 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:54.412425 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:54.412609 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:54.413211 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:54.413217 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:54.413887 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:54.414261 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:54.414281 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.415409 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.415415 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:54.416367 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:54.416380 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.417168 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.417175 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:54.417383 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:54.418052 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:54.418070 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:54.421040 (+ 2970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:54.424512 (+ 3472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:54.424517 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:54.424521 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:54.425081 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:54.425507 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:54.425512 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:54.425516 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:54.425692 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:54.426315 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:54.426320 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:54.426511 (+ 191us) client_negotiation.cc:769] Sending connection context
0811 22:24:54.426678 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 22:24:54.427032 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 22:24:54.427533 4486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.412255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54904 (local address 127.29.229.61:46837)
0811 22:24:54.412561 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:54.412566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:54.412587 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:24:54.412868 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:54.412876 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:54.412986 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:54.413165 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:54.414504 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.415244 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.416549 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.416998 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.417148 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:54.421259 (+ 4111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:54.421264 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:54.421268 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:54.421305 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:54.424327 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.425212 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.425219 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.425222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.425375 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.425819 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.425829 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.425831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.426188 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:54.426356 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:54.426901 (+ 545us) server_negotiation.cc:299] Negotiation successful
0811 22:24:54.427100 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260811 22:24:54.428645 4613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:24:54.429728 4621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.413057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57458 (local address 127.29.229.62:40533)
0811 22:24:54.413526 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:54.413534 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:54.413553 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:24:54.413650 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:54.413658 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:54.413776 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:54.413903 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:54.415460 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.416195 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.417611 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.418009 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.418152 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:54.423246 (+ 5094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:54.423251 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:54.423254 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:54.423290 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:54.426329 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.427337 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.427343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.427347 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.427522 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.428071 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.428076 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.428079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.428440 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:54.428831 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:54.429159 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 22:24:54.429420 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283}
Aug 11 22:24:54 dist-test-slave-jq4t krb5kdc[4183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487094, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.60@KRBTEST.COM for kudu/127.29.229.60@KRBTEST.COM
Aug 11 22:24:54 dist-test-slave-jq4t krb5kdc[4183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.229.61@KRBTEST.COM
I20260811 22:24:54.445256 4619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.429446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:53100)
0811 22:24:54.429621 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:54.429645 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:54.430025 (+ 380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:54.430645 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:54.430649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:54.431452 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:54.431751 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:54.431768 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.433277 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.433291 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:54.434245 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:54.434259 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.434982 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.434988 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:54.435231 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:54.435878 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:54.435895 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:54.438414 (+ 2519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:54.442501 (+ 4087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:54.442505 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:54.442507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:54.442915 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:54.443343 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:54.443359 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:54.443362 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:54.443528 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:54.444147 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:54.444152 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:54.444317 (+ 165us) client_negotiation.cc:769] Sending connection context
0811 22:24:54.444464 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 22:24:54.444816 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 22:24:54.445283 4617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.430216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53100 (local address 127.29.229.60:41389)
0811 22:24:54.430391 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:54.430395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:54.430408 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:24:54.430447 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:54.430451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:54.430508 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:54.430623 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:54.431969 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.432889 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.434410 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.434829 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.434983 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:54.438646 (+ 3663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:54.438654 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:54.438660 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:54.438700 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:54.442343 (+ 3643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.443083 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.443087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.443089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.443221 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.443680 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.443687 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.443689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.443991 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:54.444174 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:54.444604 (+ 430us) server_negotiation.cc:299] Negotiation successful
0811 22:24:54.444970 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":145,"server-negotiator.queue_time_us":80}
I20260811 22:24:54.449275 4486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.431693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54920 (local address 127.29.229.61:46837)
0811 22:24:54.431945 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:54.431950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:54.431967 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:24:54.432011 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:54.432014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:54.432086 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:54.432233 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:54.434104 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.434918 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.436436 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.437045 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.437198 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:54.443105 (+ 5907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:54.443111 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:54.443114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:54.443153 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:54.446314 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.447165 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.447172 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.447176 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.447325 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.447725 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.447731 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.447734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.448042 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:54.448347 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:54.448686 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 22:24:54.448944 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260811 22:24:54.450976 4613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20: Bootstrap starting.
I20260811 22:24:54.454648 4613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:54.454675 30612 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:24:54.456418 4613 log.cc:824] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:54.459852 4613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20: No bootstrap required, opened a new log
I20260811 22:24:54.457842 4298 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "79ca578857d54c43862e387f55a51788"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57458
I20260811 22:24:54.469751 4613 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ca578857d54c43862e387f55a51788" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } } peers { permanent_uuid: "df0c35b4e2de494280e4e2164feb9fae" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } } peers { permanent_uuid: "b8a9366146cd411180b032bd7d8dea20" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } }
I20260811 22:24:54.470204 4613 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8a9366146cd411180b032bd7d8dea20, State: Initialized, Role: FOLLOWER
I20260811 22:24:54.470695 4613 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ca578857d54c43862e387f55a51788" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } } peers { permanent_uuid: "df0c35b4e2de494280e4e2164feb9fae" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } } peers { permanent_uuid: "b8a9366146cd411180b032bd7d8dea20" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } }
I20260811 22:24:54.471818 4623 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ca578857d54c43862e387f55a51788" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } } peers { permanent_uuid: "df0c35b4e2de494280e4e2164feb9fae" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } } peers { permanent_uuid: "b8a9366146cd411180b032bd7d8dea20" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } } }
I20260811 22:24:54.472240 4623 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:24:54.473100 4613 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:24:54.483878 4639 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:24:54.484033 4639 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:24:54.481799 4491 master_runner.cc:186] Error getting master registration for 127.29.229.62:40533: OK, instance_id { permanent_uuid: "79ca578857d54c43862e387f55a51788" instance_seqno: 1786487093191437 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:24:54.494349 4491 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:24:54.506893 4474 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } attempt: 5
Aug 11 22:24:54 dist-test-slave-jq4t krb5kdc[4183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487093, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.61@KRBTEST.COM for kudu/127.29.229.60@KRBTEST.COM
I20260811 22:24:54.523229 4617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.507796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53112 (local address 127.29.229.60:41389)
0811 22:24:54.508025 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:54.508032 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:54.508059 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:24:54.508194 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:54.508199 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:54.508280 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:54.508389 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:54.509242 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.510008 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.511288 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.511803 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.511944 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:54.516060 (+ 4116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:54.516067 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:54.516072 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:54.516116 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:54.520045 (+ 3929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.521020 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.521028 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.521033 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.521192 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.521608 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.521614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.521617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.522032 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:54.522199 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:54.522597 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 22:24:54.522830 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260811 22:24:54.523229 4479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.507685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:53112)
0811 22:24:54.507902 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:54.507927 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:54.508070 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:54.508421 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:54.508426 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:54.508896 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:54.509075 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:54.509085 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.510175 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.510183 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:54.511116 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:54.511128 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.511961 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.511967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:54.512218 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:54.512936 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:54.512959 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:54.515815 (+ 2856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:54.520207 (+ 4392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:54.520213 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:54.520217 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:54.520877 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:54.521323 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:54.521329 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:54.521333 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:54.521484 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:54.522158 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:54.522163 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:54.522323 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 22:24:54.522470 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 22:24:54.522815 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 22:24:54.528712 4474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae: Bootstrap starting.
I20260811 22:24:54.532349 4474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:54.533528 4474 log.cc:824] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:54.536363 4474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae: No bootstrap required, opened a new log
I20260811 22:24:54.544399 4474 raft_consensus.cc:348] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ca578857d54c43862e387f55a51788" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } } peers { permanent_uuid: "df0c35b4e2de494280e4e2164feb9fae" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } } peers { permanent_uuid: "b8a9366146cd411180b032bd7d8dea20" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } }
I20260811 22:24:54.544831 4474 raft_consensus.cc:729] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df0c35b4e2de494280e4e2164feb9fae, State: Initialized, Role: FOLLOWER
I20260811 22:24:54.545295 4474 consensus_queue.cc:260] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ca578857d54c43862e387f55a51788" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } } peers { permanent_uuid: "df0c35b4e2de494280e4e2164feb9fae" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } } peers { permanent_uuid: "b8a9366146cd411180b032bd7d8dea20" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } }
I20260811 22:24:54.546268 4642 sys_catalog.cc:455] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ca578857d54c43862e387f55a51788" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } } peers { permanent_uuid: "df0c35b4e2de494280e4e2164feb9fae" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } } peers { permanent_uuid: "b8a9366146cd411180b032bd7d8dea20" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } } }
I20260811 22:24:54.546700 4642 sys_catalog.cc:458] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:24:54.547510 4474 sys_catalog.cc:564] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:24:54.559036 4661 catalog_manager.cc:1558] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:24:54.559345 4661 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:24:54.555337 4358 master_runner.cc:186] Error getting master registration for 127.29.229.62:40533: OK, instance_id { permanent_uuid: "79ca578857d54c43862e387f55a51788" instance_seqno: 1786487093191437 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:24:54.565989 4358 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 22:24:54.597856 4533 tablet.cc:2369] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:24:54.625180 4343 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } attempt: 5
Aug 11 22:24:54 dist-test-slave-jq4t krb5kdc[4183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487093, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.229.62@KRBTEST.COM for kudu/127.29.229.60@KRBTEST.COM
I20260811 22:24:54.645418 4617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.626406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53122 (local address 127.29.229.60:41389)
0811 22:24:54.626661 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:54.626669 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:54.626911 (+ 242us) server_negotiation.cc:407] Connection header received
0811 22:24:54.627208 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:54.627215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:54.627384 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:54.627521 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:54.629422 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.630637 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.631990 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.632766 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.632906 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:54.637346 (+ 4440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:54.637354 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:54.637361 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:54.637404 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:54.641625 (+ 4221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.642918 (+ 1293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.642923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.642926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.643101 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.643639 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.643643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.643645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.643995 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:54.644197 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:54.644662 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 22:24:54.644982 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 22:24:54.645438 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.626364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.229.60:41389 (local address 127.0.0.1:53122)
0811 22:24:54.626744 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0811 22:24:54.626799 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:24:54.626999 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:24:54.627565 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:24:54.627678 (+ 113us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:24:54.628808 (+ 1130us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:24:54.629100 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:54.629121 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.630819 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.630825 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:54.631814 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:24:54.631828 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.632926 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.632932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:24:54.633147 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:54.634203 (+ 1056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:24:54.634222 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:24:54.637051 (+ 2829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:24:54.641875 (+ 4824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:54.641882 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:54.641888 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:54.642748 (+ 860us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:54.643259 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:24:54.643264 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:24:54.643268 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:24:54.643501 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:24:54.644139 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:24:54.644144 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:24:54.644344 (+ 200us) client_negotiation.cc:769] Sending connection context
0811 22:24:54.644533 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 22:24:54.644927 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228}
I20260811 22:24:54.652519 4343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788: Bootstrap starting.
I20260811 22:24:54.656972 4343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788: Neither blocks nor log segments found. Creating new log.
I20260811 22:24:54.658535 4343 log.cc:824] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788: Log is configured to *not* fsync() on all Append() calls
I20260811 22:24:54.669140 4343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788: No bootstrap required, opened a new log
I20260811 22:24:54.678428 4343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ca578857d54c43862e387f55a51788" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } } peers { permanent_uuid: "df0c35b4e2de494280e4e2164feb9fae" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } } peers { permanent_uuid: "b8a9366146cd411180b032bd7d8dea20" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } }
I20260811 22:24:54.678892 4343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ca578857d54c43862e387f55a51788, State: Initialized, Role: FOLLOWER
I20260811 22:24:54.679441 4343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ca578857d54c43862e387f55a51788" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } } peers { permanent_uuid: "df0c35b4e2de494280e4e2164feb9fae" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } } peers { permanent_uuid: "b8a9366146cd411180b032bd7d8dea20" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } }
I20260811 22:24:54.680909 4665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ca578857d54c43862e387f55a51788" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } } peers { permanent_uuid: "df0c35b4e2de494280e4e2164feb9fae" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } } peers { permanent_uuid: "b8a9366146cd411180b032bd7d8dea20" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } } }
I20260811 22:24:54.681663 4665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:24:54.682225 4343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:24:54.682319 4298 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 22:24:54.682540 4298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:54.686923 4298 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ca578857d54c43862e387f55a51788" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } } peers { permanent_uuid: "df0c35b4e2de494280e4e2164feb9fae" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } } peers { permanent_uuid: "b8a9366146cd411180b032bd7d8dea20" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } }
I20260811 22:24:54.689838 4422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "79ca578857d54c43862e387f55a51788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "df0c35b4e2de494280e4e2164feb9fae"
I20260811 22:24:54.690382 4422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:54.690524 4568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "79ca578857d54c43862e387f55a51788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b8a9366146cd411180b032bd7d8dea20"
I20260811 22:24:54.691161 4568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:24:54.693828 4422 raft_consensus.cc:2454] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79ca578857d54c43862e387f55a51788 in term 1.
I20260811 22:24:54.694959 4243 leader_election.cc:302] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79ca578857d54c43862e387f55a51788, df0c35b4e2de494280e4e2164feb9fae; no voters:
I20260811 22:24:54.698861 4568 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79ca578857d54c43862e387f55a51788 in term 1.
I20260811 22:24:54.699873 4298 leader_election.cc:288] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [CANDIDATE]: Term 1 election: Requested vote from peers df0c35b4e2de494280e4e2164feb9fae (127.29.229.61:46837), b8a9366146cd411180b032bd7d8dea20 (127.29.229.60:41389)
I20260811 22:24:54.708277 4665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:24:54.718667 4665 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [term 1 LEADER]: Becoming Leader. State: Replica: 79ca578857d54c43862e387f55a51788, State: Running, Role: LEADER
I20260811 22:24:54.719702 4665 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ca578857d54c43862e387f55a51788" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } } peers { permanent_uuid: "df0c35b4e2de494280e4e2164feb9fae" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } } peers { permanent_uuid: "b8a9366146cd411180b032bd7d8dea20" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } }
I20260811 22:24:54.732838 4674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ca578857d54c43862e387f55a51788. Latest consensus state: current_term: 1 leader_uuid: "79ca578857d54c43862e387f55a51788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ca578857d54c43862e387f55a51788" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } } peers { permanent_uuid: "df0c35b4e2de494280e4e2164feb9fae" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } } peers { permanent_uuid: "b8a9366146cd411180b032bd7d8dea20" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } } }
I20260811 22:24:54.733605 4674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:54.740404 4707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260811 22:24:54.762138 4393 tablet.cc:2369] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:24:54.773805 4707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:24:54.783336 4729 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 22:24:54.823414 4568 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20 [term 1 FOLLOWER]: Refusing update from remote peer 79ca578857d54c43862e387f55a51788: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:24:54.824942 4422 raft_consensus.cc:1261] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae [term 1 FOLLOWER]: Refusing update from remote peer 79ca578857d54c43862e387f55a51788: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:24:54.826031 4674 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8a9366146cd411180b032bd7d8dea20" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:24:54.827903 4674 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df0c35b4e2de494280e4e2164feb9fae" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:24:54.838307 4617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.755398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53126 (local address 127.29.229.60:41389)
0811 22:24:54.755738 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:54.755747 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:54.765145 (+ 9398us) server_negotiation.cc:407] Connection header received
0811 22:24:54.765727 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:54.765734 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:54.765844 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:54.765939 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:54.769375 (+ 3436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.770599 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.784598 (+ 13999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.785348 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.785475 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:54.798559 (+ 13084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:54.798570 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:54.798576 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:54.798636 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:54.803400 (+ 4764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.810221 (+ 6821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.810232 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.810238 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.810461 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.811988 (+ 1527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.811997 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.812003 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.813440 (+ 1437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:54.813756 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:54.837044 (+ 23288us) server_negotiation.cc:299] Negotiation successful
0811 22:24:54.837777 (+ 733us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260811 22:24:54.849598 4486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.746641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54926 (local address 127.29.229.61:46837)
0811 22:24:54.746847 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:54.746855 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:54.764610 (+ 17755us) server_negotiation.cc:407] Connection header received
0811 22:24:54.768445 (+ 3835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:54.768453 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:54.768573 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:54.768655 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:54.773179 (+ 4524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.774340 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.779975 (+ 5635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.780805 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.780960 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:54.804651 (+ 23691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:54.804660 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:54.804668 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:54.804719 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:54.808983 (+ 4264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.811120 (+ 2137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.811129 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.811135 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.811328 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.814280 (+ 2952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.814288 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.814293 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.814730 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:54.815010 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:54.848336 (+ 33326us) server_negotiation.cc:299] Negotiation successful
0811 22:24:54.849166 (+ 830us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:24:54.858619 4621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:54.744941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57474 (local address 127.29.229.62:40533)
0811 22:24:54.745177 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:54.745184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:54.764185 (+ 19001us) server_negotiation.cc:407] Connection header received
0811 22:24:54.765198 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:54.765205 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:54.765302 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:54.765399 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:54.782073 (+ 16674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.783311 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.793988 (+ 10677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:54.794761 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:54.794887 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:54.815814 (+ 20927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:24:54.815825 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:24:54.815834 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:24:54.815898 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:24:54.825166 (+ 9268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.853904 (+ 28738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.853913 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.853922 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.854139 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:24:54.855551 (+ 1412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:24:54.855557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:24:54.855560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:24:54.856039 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:24:54.856352 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:54.857872 (+ 1520us) server_negotiation.cc:299] Negotiation successful
0811 22:24:54.858168 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:24:54.881762 4623 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ca578857d54c43862e387f55a51788. Latest consensus state: current_term: 1 leader_uuid: "79ca578857d54c43862e387f55a51788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ca578857d54c43862e387f55a51788" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } } peers { permanent_uuid: "df0c35b4e2de494280e4e2164feb9fae" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } } peers { permanent_uuid: "b8a9366146cd411180b032bd7d8dea20" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } } }
I20260811 22:24:54.882155 4623 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:24:54.892516 4733 mvcc.cc:205] Tried to move back new op lower bound from 7317451140380024832 to 7317451139995189248. Current Snapshot: MvccSnapshot[applied={T|T < 7317451140380024832}]
I20260811 22:24:54.892839 4642 sys_catalog.cc:455] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ca578857d54c43862e387f55a51788. Latest consensus state: current_term: 1 leader_uuid: "79ca578857d54c43862e387f55a51788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ca578857d54c43862e387f55a51788" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } } peers { permanent_uuid: "df0c35b4e2de494280e4e2164feb9fae" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } } peers { permanent_uuid: "b8a9366146cd411180b032bd7d8dea20" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } } }
I20260811 22:24:54.893184 4642 sys_catalog.cc:458] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:24:54.895761 4707 catalog_manager.cc:1347] Generated new cluster ID: af7ab5ce6ec74ea8acf5d243bcfc8e8f
I20260811 22:24:54.895555 4665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "79ca578857d54c43862e387f55a51788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ca578857d54c43862e387f55a51788" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } } peers { permanent_uuid: "df0c35b4e2de494280e4e2164feb9fae" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } } peers { permanent_uuid: "b8a9366146cd411180b032bd7d8dea20" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } } }
I20260811 22:24:54.895892 4707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:24:54.900906 4623 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "79ca578857d54c43862e387f55a51788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ca578857d54c43862e387f55a51788" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } } peers { permanent_uuid: "df0c35b4e2de494280e4e2164feb9fae" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } } peers { permanent_uuid: "b8a9366146cd411180b032bd7d8dea20" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } } }
I20260811 22:24:54.901190 4623 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:24:54.895902 4665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:54.929766 4665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "79ca578857d54c43862e387f55a51788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ca578857d54c43862e387f55a51788" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } } peers { permanent_uuid: "df0c35b4e2de494280e4e2164feb9fae" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } } peers { permanent_uuid: "b8a9366146cd411180b032bd7d8dea20" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } } }
I20260811 22:24:54.930119 4665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788 [sys.catalog]: This master's current role is: LEADER
I20260811 22:24:54.934942 4642 sys_catalog.cc:455] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "79ca578857d54c43862e387f55a51788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79ca578857d54c43862e387f55a51788" member_type: VOTER last_known_addr { host: "127.29.229.62" port: 40533 } } peers { permanent_uuid: "df0c35b4e2de494280e4e2164feb9fae" member_type: VOTER last_known_addr { host: "127.29.229.61" port: 46837 } } peers { permanent_uuid: "b8a9366146cd411180b032bd7d8dea20" member_type: VOTER last_known_addr { host: "127.29.229.60" port: 41389 } } }
I20260811 22:24:54.935240 4642 sys_catalog.cc:458] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:24:54.946133 4707 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:24:54.948079 4707 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:24:54.965346 4707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79ca578857d54c43862e387f55a51788: Generated new TSK 0
I20260811 22:24:54.966591 4707 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:24:55.096889 30612 mini_kdc.cc:348] Time spent kdestroy: real 0.107s user 0.000s sys 0.039s
W20260811 22:24:55.117333 4617 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:55.101894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53142 (local address 127.29.229.60:41389)
0811 22:24:55.102103 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:55.102110 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:55.114948 (+ 12838us) server_negotiation.cc:407] Connection header received
0811 22:24:55.115427 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:55.115434 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:55.115540 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:55.115621 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:55.117044 (+ 1423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53142: BlockingRecv error: recv got EOF from 127.0.0.1:53142 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 22:24:55.118963 4621 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:55.102191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57488 (local address 127.29.229.62:40533)
0811 22:24:55.102580 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:55.102588 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:55.114414 (+ 11826us) server_negotiation.cc:407] Connection header received
0811 22:24:55.117142 (+ 2728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:55.117148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:55.117241 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:55.117316 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:55.118586 (+ 1270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57488: BlockingRecv error: recv got EOF from 127.0.0.1:57488 (error 108)
Metrics: {"server-negotiator.queue_time_us":190}
W20260811 22:24:55.119165 4486 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:55.102766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54938 (local address 127.29.229.61:46837)
0811 22:24:55.103065 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:55.103075 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:55.117038 (+ 13963us) server_negotiation.cc:407] Connection header received
0811 22:24:55.117297 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:55.117303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:55.117431 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:55.117513 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:55.118918 (+ 1405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54938: BlockingRecv error: recv got EOF from 127.0.0.1:54938 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 22:24:55.126297 4486 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:55.123675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54946 (local address 127.29.229.61:46837)
0811 22:24:55.123848 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:55.123854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:55.124353 (+ 499us) server_negotiation.cc:407] Connection header received
0811 22:24:55.124714 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:55.124721 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:55.124862 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:55.124944 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:55.126004 (+ 1060us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54946: BlockingRecv error: recv got EOF from 127.0.0.1:54946 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 22:24:55.139389 4486 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:55.136641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54962 (local address 127.29.229.61:46837)
0811 22:24:55.136832 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:55.136839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:55.137400 (+ 561us) server_negotiation.cc:407] Connection header received
0811 22:24:55.137725 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:55.137731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:55.137829 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:55.137908 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:55.139121 (+ 1213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54962: BlockingRecv error: recv got EOF from 127.0.0.1:54962 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 22:24:55.148729 4486 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:55.146048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54976 (local address 127.29.229.61:46837)
0811 22:24:55.146219 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:55.146225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:55.146717 (+ 492us) server_negotiation.cc:407] Connection header received
0811 22:24:55.147006 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:55.147012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:55.147121 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:55.147199 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:55.148372 (+ 1173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54976: BlockingRecv error: recv got EOF from 127.0.0.1:54976 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 22:24:55.160581 4486 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:55.158152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54984 (local address 127.29.229.61:46837)
0811 22:24:55.158345 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:55.158352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:55.158908 (+ 556us) server_negotiation.cc:407] Connection header received
0811 22:24:55.159121 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:55.159129 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:55.159256 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:55.159389 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:55.160295 (+ 906us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54984: BlockingRecv error: recv got EOF from 127.0.0.1:54984 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 22:24:55.175266 4486 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:55.172272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55000 (local address 127.29.229.61:46837)
0811 22:24:55.172526 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:55.172532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:55.173256 (+ 724us) server_negotiation.cc:407] Connection header received
0811 22:24:55.173660 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:55.173671 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:55.173806 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:55.173990 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:55.174996 (+ 1006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55000: BlockingRecv error: recv got EOF from 127.0.0.1:55000 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 22:24:55.198390 4486 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:55.195897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55016 (local address 127.29.229.61:46837)
0811 22:24:55.196113 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:55.196120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:55.196751 (+ 631us) server_negotiation.cc:407] Connection header received
0811 22:24:55.197061 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:55.197068 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:55.197167 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:55.197281 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:55.198133 (+ 852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55016: BlockingRecv error: recv got EOF from 127.0.0.1:55016 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 22:24:55.237581 4486 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:55.235215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55018 (local address 127.29.229.61:46837)
0811 22:24:55.235502 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:55.235513 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:55.236086 (+ 573us) server_negotiation.cc:407] Connection header received
0811 22:24:55.236257 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:55.236264 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:55.236369 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:55.236524 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:55.237292 (+ 768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55018: BlockingRecv error: recv got EOF from 127.0.0.1:55018 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 22:24:55.307420 4486 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:55.305268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37820 (local address 127.29.229.61:46837)
0811 22:24:55.305520 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:55.305528 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:55.305835 (+ 307us) server_negotiation.cc:407] Connection header received
0811 22:24:55.306001 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:55.306008 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:55.306112 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:55.306253 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:55.307157 (+ 904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37820: BlockingRecv error: recv got EOF from 127.0.0.1:37820 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
W20260811 22:24:55.442418 4486 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:24:55.440234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37824 (local address 127.29.229.61:46837)
0811 22:24:55.440427 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:55.440432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:55.440903 (+ 471us) server_negotiation.cc:407] Connection header received
0811 22:24:55.441077 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:55.441081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:55.441157 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:55.441259 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:24:55.442128 (+ 869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37824: BlockingRecv error: recv got EOF from 127.0.0.1:37824 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:24:55.486192 4639 catalog_manager.cc:1259] Loaded cluster ID: af7ab5ce6ec74ea8acf5d243bcfc8e8f
I20260811 22:24:55.486341 4639 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20: loading cluster ID for follower catalog manager: success
I20260811 22:24:55.489073 4639 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20: acquiring CA information for follower catalog manager: success
I20260811 22:24:55.490975 4639 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b8a9366146cd411180b032bd7d8dea20: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:24:55.561542 4661 catalog_manager.cc:1259] Loaded cluster ID: af7ab5ce6ec74ea8acf5d243bcfc8e8f
I20260811 22:24:55.561653 4661 catalog_manager.cc:1552] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae: loading cluster ID for follower catalog manager: success
I20260811 22:24:55.564201 4661 catalog_manager.cc:1574] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae: acquiring CA information for follower catalog manager: success
I20260811 22:24:55.565996 4661 catalog_manager.cc:1602] T 00000000000000000000000000000000 P df0c35b4e2de494280e4e2164feb9fae: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:24:55.712687 4486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:55.706506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37836 (local address 127.29.229.61:46837)
0811 22:24:55.706770 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:55.706777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:55.706993 (+ 216us) server_negotiation.cc:407] Connection header received
0811 22:24:55.707077 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:55.707082 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:55.707332 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:55.707442 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:24:55.707848 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:55.708638 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:55.710274 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:55.710690 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:55.710826 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:55.711404 (+ 578us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:24:55.711964 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:24:55.712097 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:55.712231 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 22:24:55.712310 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260811 22:24:55.726090 4866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:55.719926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40362 (local address 127.29.229.60:41389)
0811 22:24:55.720437 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:55.720442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:55.720557 (+ 115us) server_negotiation.cc:407] Connection header received
0811 22:24:55.720797 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:55.720800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:55.720970 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:55.721120 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:24:55.721531 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:55.722262 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:55.723595 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:55.724107 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:55.724211 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:55.724778 (+ 567us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:24:55.725375 (+ 597us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:24:55.725488 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:55.725619 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 22:24:55.725733 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":270,"threads_started":1}
I20260811 22:24:55.750682 4875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:55.732504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54646 (local address 127.29.229.62:40533)
0811 22:24:55.744308 (+ 11804us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:55.744316 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:55.744341 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:24:55.744509 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:55.744516 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:55.744753 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:55.744953 (+ 200us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:24:55.745553 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:55.746458 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:55.748364 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:55.748872 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:55.749017 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:55.749507 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:24:55.749949 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:24:55.750065 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:55.750258 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 22:24:55.750355 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11659,"thread_start_us":11546,"threads_started":1}
I20260811 22:24:55.751359 4866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:55.733128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40366 (local address 127.29.229.60:41389)
0811 22:24:55.738504 (+ 5376us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:55.738511 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:55.744790 (+ 6279us) server_negotiation.cc:407] Connection header received
0811 22:24:55.744987 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:55.744993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:55.745102 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:55.745196 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:24:55.746084 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:55.746933 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:55.749110 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:55.749754 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:55.749824 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:55.750438 (+ 614us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:24:55.750614 (+ 176us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:24:55.750683 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:55.750974 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 22:24:55.751057 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5199}
I20260811 22:24:55.751992 4486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:24:55.733368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37840 (local address 127.29.229.61:46837)
0811 22:24:55.733606 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 22:24:55.733615 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:24:55.745137 (+ 11522us) server_negotiation.cc:407] Connection header received
0811 22:24:55.745375 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:24:55.745380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:24:55.745495 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:24:55.745571 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:24:55.746353 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:55.747485 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:55.749686 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:24:55.750329 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:24:55.750445 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:24:55.751169 (+ 724us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:24:55.751301 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:24:55.751378 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:24:55.751607 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 22:24:55.751689 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:24:55.756587 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 4209
I20260811 22:24:55.758177 4334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:24:55.889789 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 4209
I20260811 22:24:55.925495 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 4358
I20260811 22:24:55.926872 4464 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:24:56.040371 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 4358
I20260811 22:24:56.071239 30612 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu pid 4491
I20260811 22:24:56.072727 4604 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:24:56.193281 30612 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskD_Yjlj/build/asan/bin/kudu with pid 4491
2026-08-11T22:24:56Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5319 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10772 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (411052 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST